NAB Show: Hammerspace Expands Support for Commodity Hardware in Global Data Environment
Introduces high-performance erasure coding to add efficiency, data durability and HA.
This is a Press Release edited by StorageNewsletter.com on April 18, 2024 at 2:02 pmHammerspace, Inc. announced the addition of high-performance erasure coding to its Global Data Environment software.
The company unifies data stored on any vendor’s existing NAS, object or cloud storage and this announcement further expands the range of storage options that the firm supports. With the addition of high-performance erasure coding, it offers its customers a new option for building highly resilient storage environments with commodity off-the-shelf hardware.
“This announcement is the latest in a series of Hammerspace innovations that are radically changing how data is used and preserved,” said David Flynn, founder and CEO. “We started this year by adding support for data on tape, then pioneered the Hyperscale NAS architecture for AI and GPU computing, and now we are further expanding our data storage services with high-performance erasure coding. This gives customers even more choice and flexibility when it comes to their storage infrastructure.”
Click to enlarge
Erasure coding gives option for building high-performance storage clusters
The company remains a 1st and only Global File System that supports storage systems from any data center storage or cloud vendor such as Infinidat, Pure Storage, Cloudian, AWS, and countless others – including off-the-shelf storage servers, flash, NAS, object, block, cloud, and even tape. Its innovative architecture makes data from existing siloed storage systems globally visible and delivers high-performance throughput, IO/s, and metadata Ops for both reads and writes to compute environments, applications, and users in minutes – simplifying everything from data migration to assembling datasets for AI training from distributed data sources.
Hammerspace erasure coding groups (EC-Groups), built on the Mojette Transform erasure code, is resilient, efficient, and highly performant – up to 2x faster than traditional erasure coding schemes – and extends the suite of enterprise data services offered by the company.
This gives customers a new option for building high-performance storage using the firm’s technology on commodity hardware. For example when building scratch space for HPC and research environments, using OCP hardware in hyperscale environments, or for building out a new, all-flash storage environment for AI initiatives to augment or replace legacy storage.
Benefits of mojette transform erasure code
The company has implemented the Mojette Transform erasure code as a data storage service within the Global Data Environment. Mojette Transform is an extremely CPU-efficient erasure code which avoids the complex mathematical calculations of usual erasure codes, leaving more compute power available to applications.
The Mojette Transform erasure code is optimal for the high performance Hammerspace provides, not only in large file throughput but also in delivering the extreme levels of write IO/s by avoiding the ‘read-modify-write’ needed by other erasure codes, which causes them to struggle with IO/s performance.
“Our goal in developing the Mojette Transform erasure code was to deliver the highest reliability in data protection, coupled with extreme performance, leveraging commodity hardware,” said Pierre Evenou, VP advanced technology, Hammerspace, and founding engineer of Mojette Transform. “The result is delivering close to native performance of the underlying storage hardware to the application and compute environment without sacrificing data protection.”
Resources:
Hammerspace Global Data Environment: Product overview
Hammerspace for High-Performance and GPU Computing
Comments
This is a major product iteration for Hammerspace with a real IP on inline data protection providing a key element in the global data management chain. It will help Linux-based file servers with local storage to increase their resiliency and durability while reducing cost. With the Mojette Transform implementation, the protection delivers an interesting storage hardware efficiency 66% or an capacity overhead of 50% with very limited CPU and load impacts. The erasure coding is completed by CRC to add some integrity verification mechanisms.
The performance offers 2x better rates than Reed-Solomon implementation. The design is highly parallel with a fine granularity of 4KB, aligned with the Linux page size.
A minimum of 4 nodes must be configured with some flexibility in 2 directions, addition of disks for a scale-up model and/or nodes for scale-out and of course a mix of the 2.
Click to enlarge
Subscribe to our free daily newsletter
