Scality to Reveal Big Data Solution Available for Hadoop

Scality announced a milestone for the storage industry, with the first production-grade implementation of Apache Hadoop on Cloud Data Management Interface (CDMI).

Storage implementation for Hadoop delivers benefits awaited by the Hadoop community, resulting in a cost effective, easier-to-use, more resilient and higher performing Hadoop infrastructure.

• Eliminates the single-point-of-failure in Hadoop’s architecture by replacing Hadoop’s NameNode server with Scality’s own metadata architecture.
• Enables compute on the storage node itself, reducing the need for data transfer by enabling in-place processing and sharing data location with the Job Tracker.
• Delivers high levels of data protection without the excessive hardware overhead of standard replication by making available its implementation of erasure coding technology (ARC).
• Eliminates the need to load files through HDFS by utilizing Scality’s Open Cloud Access (OCA): users can read and write files through a standard file system and at the same time process the content with Hadoop. In addition they benefit from the scalability of Scality’s SOFS (Scale-Out File System)

"Hadoop is an amazing set of tools to analyze large amounts of data relatively simply. Over the past two years, we have seen a growing demand from our customers for help solving the inefficiencies of the existing open source distributions. We have contributed our Hadoop solution to the CDMI community, thus ensuring that it can be used with any CDMI-compatible storage. However when it is deployed on Scality, users get the added benefits of RING’s self-healing, efficient data protection, and unsurpassed performance," says Jérôme Lecat, CEO of Scality. "Furthermore, our OCA strategy makes it even easier for service providers to use Hadoop without changing their application, since our CDMI framework can read data directly from the file system."

Scality does not intend to replace or compete with existing Hadoop distributions. Instead, it expects to enhance existing open-source projects by delivering better storage solutions. Scality’s storage for Hadoop is compatible  and has been tested with Hortonworks HDP 1.0 and Cloudera CDH4.

"Our engineering team did a fantastic job leveraging CDMI and SOFS as key components to boost the Hadoop experience," said Philippe Nicolas, director of product strategy at Scality. "Scality’s RING storage software is a very comprehensive solution. Its tremendous degree of concurrency delivers unbeatable throughput and resilience in this context."