Dropbox Develops Future of Cloud Infrastructure With SMR Technology Deployment For Magic Pocket

Dropbox, Inc. announced a new chapter in the evolution of Magic Pocket, its custom-built storage infrastructure.

The company is deploying Shingled Magnetic Recording (SMR) drive technology, which will increase overall storage density, reduce the company’s physical data center footprint, and provide cost savings without sacrificing performance or reliability. The company will be the first company to test and deploy SMR technology at this scale.

The firm anticipates approximately a quarter of its Magic Pocket infrastructure will be on SMR drive capacity by 2019. In the coming months, the company also plans to open source the test software it created in the process, which will let other companies qualify SMR technology for their own enterprise storage environments.

Magic Pocket architecture
Click to enlarge

“Creating our own storage infrastructure was a huge technological challenge, but it’s already paid dividends for our customers and our business,” said Quentin Clark, SVP, engineering, product, and design, Dropbox. “As more teams adopt Dropbox, SMR technology will help us scale our infrastructure by providing greater flexibility, efficiency, and cost savings. We’re also excited to make this technology open-source so other companies can benefit from it.“

The move to SMR technology has been a significant undertaking: the company sourced SMR disk drives from third-party suppliers, designed a bespoke hardware and component ecosystem around it, and created new software to ensure its compatibility with existing Magic Pocket architecture. The company ultimately chose SMR technology for its ability to expand storage capacity from 8 to 14TB per disk, while delivering the performance and reliability the company expects.

This process is another example of the company’s engineering expertise in building large distributed systems. By the end of 2018, the firm will have an infrastructure footprint spanning 29 facilities in twelve countries and four continents, including storage for users inside and outside the U.S.

Expanding impact of Magic Pocket
In 2016, Magic Pocket set a standard in the industry by introducing an exabyte-scale storage infrastructure that was reliable and secure, and could meet the dynamic workloads of more than 500 million registered users. The system was designed to provide annual data durability of over 99.9999999999%, and availability of over 99.99%. This move to SMR technology is the next phase in the company’s development of its custom-built infrastructure.

Resources:
Blog: H ow Dropbox engineering is successfully deploying SMR disk drives at scale in Magic Pocket
Blog: H ow Dropbox created Magic Pocket