Hadoop Storage Acceleration and Optimization Platform

Altior Inc. announces the availability of AltraSTAR – Hadoop Storage Accelerator and Optimizer, based on its Altraflex HW acceleration platform and CeDeFS Filter Layer software.

AltraSTAR can reduce Map/Reduce job execution time, increase data node storage capacity up to 6x, while simultaneously delivering increased I/O performance.

In a Terasort test using an eight data node Hadoop cluster with dual six-core CPUs per node, AltraSTAR was able to reduce storage consumption by half, and improve Map/Reduce job execution time by 15%. As a result, end-users can see a drop in the 3-year Hadoop cluster by as much as 30%.

This TCO benefit is derived from:

• Lowest Capital Expenditure: Since storage is the most expensive component of the system reducing storage requirements by half in data nodes will reduce initial system cost.
• Highest Rack Analytics Density: By optimizing storage capacity, the Hadoop Acceleration Platform provides high processing power and analytics density in a given rack. It provides great data analytics density per square foot of any Hadoop cluster – about 1PB per rack for fully populated data nodes.
• Faster Data Analytics: Most Hadoop jobs are disk IO bound, and AltraSTAR improves disk IO performance. This Reduced Power Consumption: Altior’s FPGA based HW Acceleration consume up to 90% less power than CPU-based solutions, resulting in savings in the expense of operating the cluster.
• Improved Reliability: Storage devices, because of moving parts, hinder reliability. By halving the storage disks per node, Hadoop Acceleration solutions increase the nodes MTBF for a given amount of storage.

Hadoop jobs must process huge amounts of data. Efficient processing requires that as much data as possible be loaded into the Hadoop data nodes. Altior’s CeDeFS data optimization system ensures the efficient use of all available storage. Hadoop data is usually quite compressible consisting of text files, server logs, and click streams. CeDeFS’s data compression can optimize available storage on the data nodes by as much as six-fold. In addition to enlarged storage capacity, it also provides an I/O acceleration service to the Hadoop cluster. Since data is compressed on disk it can be delivered more quickly to I/O bound processing tasks. The result for the Hadoop user is faster Map/Reduce execution times since the cluster’s processors will spend less time waiting idle for I/O completion and more time on application job execution. The I/O acceleration effect of CeDeFS data optimization will generally exceed that possible from SSDs at a small fraction of the cost and over many times the capacity of an SSD.

AltraSTAR consists of the CeDeFS filter layer software for Linux and the AltraFlex data compression accelerator hardware. The AltraFlex compression accelerator delivers over 10Gbps of compression throughput while consuming less than 10W of power. Additional acceleration cards can be added to scale throughput even further. Compared to a multi core server class CPU, the AltraFlex compression accelerator delivers as much as 4.25 times more compression throughput while consuming less than 1/8th the power.

AltraSTAR can be integrated with any standard Map/Reduce solutions such as Cloudera’s CDH3 and CDH4 – without requiring any change in the Hadoop application or workflow. The AltraFlex compression accelerator operates asynchronously in parallel with the host CPU. The combination of asynchronous operations and high speed compression means that CeDeFS will operate without adding any latency to the disk I/O subsystem.

"Data optimization provides benefits to Hadoop users that go well beyond simple data compression," said Ramana Jampala, Altior CEO. "With greater storage capacity larger jobs can be run on fewer nodes. Furthermore, since AltraSTAR accelerates I/O, jobs will run faster while consuming less power."

AltraSTAR for Hadoop is available. Accelerator pricing depends on the exact configuration of the accelerator card and acceleration cores.