DDN and IBM Join Forces to Deploy Active Archive System for Yahoo Japan

DataDirect Networks, Inc. (DDN) announced that Yahoo Japan Corporation (Yahoo! JAPAN) has deployed an active archive system jointly developed by DDN and IBM Japan.

The system allows Yahoo! JAPAN to cache dozens of petabytes of data from its OpenStack Swift storage solution in a Japan-based data center, and transfer data to a US-based data center at a rate of 50TB of data per day – thus enabling energy cost savings of 74% due to lower energy rates in the US versus Japan, while ensuring fast data access regardless of location.

Yahoo! JAPAN introduced the active archive system, which allows data to be cached in a data center in Japan and then transferred and stored in a US-based data center owned by YJ America, Inc., its local subsidiary in the US. The decision to move to an active archive system configured across dual continents was based, in large part, on Yahoo! JAPAN’s rising data volumes, multi-petabyte backup requirements, and the DR measures it implemented after the Great East Japan Earthquake.

The joint DDN/IBM Japan system is able to handle dozens of petabytes of data within a single file system configuration. It allows the data center to cache data from an operating object storage (OpenStack Swift) private cloud at a rate of 11TB/ hour and backup the data in a data center archive in the US at a transfer rate of 50TB per day. The system allows users in Japan to access data and conduct services easily and without concern for where the data resides.

The active archive is built on DDN’s SFA7700X storage solution and IBM Spectrum Scale, which together provide the I/O performance required. The data is cached on a SSD that enables higher I/O and ensures expedited metadata handling. The solution enables data sharing by using the caching function via the Scale Active File Management (AFM). In addition, the high density SFA7700X ensures capacity and scalability to meet Yahoo! JAPAN’s fast-growing, high-volume needs.

“At Yahoo! JAPAN, we were grappling with a number of challenges related to our large, fast-paced data growth and vital DR needs; however, installing a massive storage system in a data center in Japan raises additional issues such as power consumption,” said Daisuke Masaki, cloud innovations, site operations division, systems management group, Yahoo Japan. “We therefore opted for a bold technical solution in which our data center in Japan caches data from the existing object storage (OpenStack Swift) and saves the data to a data center archive in the United States, which can be operated with 26% of the electricity cost of a data center in Japan and at about one-third of the cost of competitive solutions. Moving forward, we plan to expand and save data from multiple websites in Japan to the active archive system in the United States.“