SC23: Linux Foundation Announced Distributed Asynchronous Object Storage Foundation

The Linux Foundation announced DAOS Foundation, an initiative aimed at advancing the governance and development of the Distributed Asynchronous Object Storage (DAOS) project.

Featuring support from founding members Argonne National Laboratory (ANL), Enakta Labs, Google Cloud, Hewlett Packard Enterprise (HPE), and Intel Corp., this milestone represents a step toward accelerating next-gen HPC, and AI/ML, with performance.

The DAOS is a high performance open source storage system that has redefined performance for a wide spectrum of HPC and AI workloads. It has achieved this feat through the creation of a fully distributed key-value storage architecture that successfully mitigates the data and metadata bottlenecks commonly associated with traditional POSIX-based HPC storage systems. By harnessing the power of commodity hardware, It optimizes both performance and cost, supporting file, block and object semantics and a diverse range of application interfaces, including TensorFlow, HDF5 and MPI-IO. Advanced features such as data protection with self-healing, end-to-end data integrity, and fine-grained access control have made DAOS an industry leader.

Industry leaders come together and endorse DAOS storage software to accelerate nextgen HPC and AI/ML workloads. 

Starting in 2012, Intel has been a key contributor to DAOS source code, which has since grown into an established open source community spanning various hardware and software on-prem and cloud vendors. To provide a broader governance to the project and to foster future development and growth, the DAOS Foundation has launched as a project under the umbrella of the Linux Foundation. This will ensure the continuous development of DAOS as an open source project and will enable other participants to engage in architectural and development decisions. Intel will donate the DAOS source code to the DAOS Foundation and will remain one of the main contributors to the future development of the project.

“The DAOS storage system stack provides the core high-performance storage system for Argonne Leadership Computing Facilities latest supercomputer, Aurora,” said Kevin Harms, ALCF-4 technical director, Argonne National Laboratory. “DAOS has performance 20x greater than our current file systems and will provide better performance across a broad range of I/O workload allowing application science codes to achieve high performance with less effort.“

“Enakta Labs takes great pride in its role as a founding member of the DAOS Foundation,” said Denis Nuja, CEO, Enakta Labs. “We consider it a privilege to actively contribute and collaborate with the open source community in advancing next-generation storage technology. DAOS serves as the foundation for the Enakta Platform, and we are excited to introduce this cutting-edge, high performance storage system to the market.“

“Google Cloud is delighted to be a founding member of the DAOS Foundation,” said Sameet Agarwal, storage VP/GM, Google Cloud. “This world record-setting technology underpins our new storage service, Parallelstore, and we are extremely excited to accelerate the growth of HPC and AI/ML workloads in the cloud.“

“HPE is honored to be a founding member of the DAOS Foundation and have this opportunity to contribute to a community that values open standards,” said Andrew Wheeler, HPE Fellow,VP and director, Hewlett Packard Labs. “HPE has been closely engaged with Intel on Argonne National Laboratory’s exascale deployment of DAOS and continues to contribute to the community. We support the DAOS architecture, which we believe will advance emerging supercomputing and technical computing use cases.”

“Intel is taking a pivotal step forward, solidifying our dedication to open source technology,” said Hillarie Prestopine, VP, system software, data center AI solutions, Intel. “As a founding member of the DAOS Foundation, we are reinforcing our commitment to open source software and providing a platform for the expansion of software development surrounding DAOS. The DAOS Foundation will drive innovation in storage software, delivering what is essential for AI and high performance computing.”

The DAOS Foundation is open to anyone willing to contribute to the project. It offers Premier, General, and Associate membership levels, each providing a different level of participation in governing board and technical working groups. To learn more about the DAOS Foundation, including how to become a member and join their community, visit the DAOS Foundation website.

Resources:
Welcome to DAOS Version 2.4
DAOS overview