IBM Combines All-Flash and Storage Software for Hortonworks

IBM Corporation announced an all-flash, performance data and file management solution for enterprise clients running exabyte-scale big data analytics, cognitive and AI applications.

The combined flash and storage software solution has been certified with the Hortonworks Data Platform (HDP) to provide clients with more choice in selecting the right platform for their big data analytics on data processing engines like Hadoop and Spark.

The company’s All-Flash Elastic Storage Server (ESS), powered by Spectrum Scale exabyte-scale software-defined storage, increases data throughput by 60% ⁽¹⁾ over previous solutions. Performance improvements produce a storage solution powerful enough to run complex applications from machine and deep learning clusters to image recognition of cancer cells.

The Spectrum Scale storage software combines data from silos across the enterprise into a single view in order to run analytics on Hadoop and Spark. Through unifying file, object and Hadoop HDFS storage, ESS with Spectrum Scale combines to shorten time to results while contributing to reducing storage capacity requirements by more than 55% ⁽²⁾ for Hadoop and Spark workload data. The solution provides also user data with essential features such as redundant data paths, distributed erasure coding and end-to-end checksums. Based upon millions of hours of current production systems, reported over the past two years the ESS design has delivered five nines of availability against unscheduled downtime.

“We’re seeing an increasing appetite on the part of clients to unlock value within their own business data where competitive advantage can be found but too often they face challenges that inhibit ready access to this information,” said Ed Walsh, GM, storage and software defined infrastructure, IBM. “IBM ESS with Spectrum Scale has been developed to make data available by reducing obstacles that prevent users from quickly accessing and processing the growing oceans of differing data types stored within their disparate, isolated storage systems.“

The All-Flash ESS represents a new generation of data and file management. To help clients more effectively manage their hot and cold data. ESS with Spectrum Scale, an Exabyte-scale software-defined storage solution, supports a variety of network protocols while providing clients the elasticity to transparently tier data across flash, disk, tape and cloud storage systems.

With all ESS and Spectrum Scale models certified on Hortonworks, IBM fulfill a commitment to deliver higher exabyte scale storage densities and related performance throughput required for programs like the company’s support for the Department of Energy CORAL project. ESS integration with Spectrum Scale, which is being developed to support anticipated NVMe-based storage solutions from the company, sets the stage for increased proficiency in managing throughput of big data analytics workloads.

The company will be taking orders on the All-Flash Elastic Storage Server (ESS) but the solution will be available on August 25, 2017.

Blog: Fast data: The future of big data

⁽¹⁾ Comparing the IBM ESS GS6 25GB/s against the ESS GS6S 40GB/s maximum throughput
⁽²⁾ Comparing open source HDFS 3x data replication v. IBM Spectrum Scale RAID-1.22x erasure coded data protection.