Los Alamos National Laboratory and Pavilion Partnering

Pavilion Data Systems, Inc. announced a partnership with Los Alamos National Laboratories (LANL) to co-develop and evaluate acceleration of analytics by offloading analytics functions from storage servers to the storage array, minimizing data movement by enabling data reduction near the storage.

LANL is moving their I/O from file based to record or column based, which enables analytics to be done using tools from the big data/analytics community. LANL has shown 1,000x speedups on analytics functions by leveraging data reduction near the storage devices via their DeltaFS technology. Shaping standards and commercialization of storage capabilities to assist in leveraging analytics near the storage is at the heart of the partnership.

“For decades, our large-scale physics simulations used file-based I/O, as these simulations were tuned for disk-based parallel file systems,” stated Gary Grider, HPC division leader, LANL. “The file-based I/O had hidden the inherent structure in the data in those files. Switching to record/column-based I/O exposes the structure in the data, thereby enabling analytics.  Leveraging technology from the big data/analytics community is a natural method to accomplish this, and partnering with industry to explore this at extreme scale is the basis for this partnership with Pavilion.“

“We are excited about partnering with LANL on this high-performance innovation, as together we explore the use of HyperOS functionality to enable offloaded analytics from extreme-scale scientific applications,” stated Dario Zamarian, Pavilion CEO.

The data processing algorithms of Pavilion HyperOS coupled with the performance density of the Pavilion HyperParallel Flash Array provides a performant computational storage array capability enabling analytics offloads at scale.