Sorting 1.5TB of Data in 60 Seconds

MapR Technologies, Inc., in Hadoop technology, announced at the O’Reilly Strata Conference a record for MinuteSort, sorting 1.5TB records in 60 seconds using Google Compute Engine and the MapR Distribution for Apache Hadoop.

The benchmark, often referred to as the World Cup of data sorting, demonstrates how quickly data can be sorted starting and ending on disks.

MapR’s record-setting benchmark was completed on 2,103 virtual instances in the Google Compute Engine. Each virtual instance consisted of four virtual cores and one virtual disk, for a total of 8,412 virtual cores and 2,103 virtual disks. The new record surpassed the previous record of 1.4TB. It is also almost three times the amount of data processed by the previous Hadoop MinuteSort record which was 578GB.

"The record is significant because it represents a total efficiency improvement executed in a cloud environment," said Jack Norris, VP of marketing, MapR Technologies. "In an era where information is increasing by tremendous leaps, being able to quickly scale to meet data growth with high performance makes business analytics a reality in situations previously impossible."

While the previous MinuteSort record was achieved with custom hardware, MapR set the record using commercially available Google Compute Engine, Hadoop MapReduce and the MapR Distribution. Google Compute Engine is currently in Limited Preview but will soon be available so any business or developer can benefit from the scale, performance and value of Google’s infrastructure.

Combined with data protection and BC, MapR enables customers to harness the power of big data analytics. Companies including Amazon, Cisco, EMC and Google partner with MapR to deliver an enterprise-grade Hadoop solution. Investors include Lightspeed Venture Partners, NEA and Redpoint Ventures.