DDN Infinite Memory Engine Burst Buffer Exceeds 1TB/s File System Performance

DataDirect Networks, Inc. (DDN) announced that Japan’s highest performance supercomputer has achieved effective I/O performance exceeding 1TB/s using the company’s Infinite Memory Engine (IME) to deliver a high speed file cache system.

“Storage performance has been one of the biggest challenges in developing supercomputers. To meet the demands for storage performance, IME was introduced to the Oakforest-PACS on a massive scale, the first such introduction in the world,” said Osamu Tatebe, lead, public relations, JCAHPC/professor, Center for Computational Sciences, University of Tsukuba. “We are very pleased that we could achieve effective I/O performance exceeding 1TB/s in writing tens of thousands of processes to the same file. With this new storage technology, we believe that we will be able to contribute to society with the further development of computational science, big data analysis and machine learning.“

The Oakforest-PACS massively parallel supercomputer is operated by the Joint Center for Advanced HPC (JCAHPC), which is run collaboratively by the Information Technology Center at the University of Tokyo and the Center for Computational Sciences at the University of Tsukuba. With resources for joint use, Oakforest-PACS will advance R&D in a variety of science and technology fields. It will be used for computational science research, as well as to develop talent in the computational science and HPC fields.

To measure effective I/O performance, DDN used IOR, the I/O throughput benchmark published by Livermore Computing Center of Lawrence Livermore National Laboratory, and achieved more than 1TB/s in both I/O access patterns: FPP (File Per Process where parallel processes perform independent file I/O respectively) and SSF (Single Shared File where all parallel processes perform I/O to a single shared file).

SSF is an access method that cannot realize sufficient performance with a conventional parallel file system, but is considered an effective access method for the next generation of exascale supercomputers. With its marks for effective performance across FPP and SSF I/O access patterns, company’s IME has proven its eminence as an I/O system for conventional I/O processing, and also for future exascale supercomputing. Results of the IOR benchmark are shown below:

The file system cache of the Oakforest-PACS comprises 25 IME14KX systems. One IME14KX has 48x800GB NVMe SSDs and eight Intel Omni-Path ports. The 25 systems provide a cache of 960TB and a logical bandwidth of 1.5TB/s within a space as small as 100 rack units, which is about 2 ½ data center racks. This is one-tenth to one one-hundredth the amount of equipment required to support other current TB/s file systems.

“We are very excited to work with JCAHPC and Fujitsu Limited on innovative, next-generation flash-based storage technologies for scientific computing and big data, and we regard the Oakforest-PACS project as a true partnership between users and suppliers that will advance the way we think about data and storage in high-performance computing,” said Robert Triendl, SVP, global sales, marketing and field services, DDN.

Resources:
The Joint Center for Advanced High Performance Computing (JCAHPC)
The Information Technology Center, University of Tokyo
The Center for Computational Sciences, University of Tsukuba