YCSB Benchmark From Thumbtack Technology

The advantage of flash technology for powering fast big data transactions is highlighted in a new independent NoSQL database benchmark report from Thumbtack Technology, Inc.

Aerospike, Inc. announced that the Thumbtack benchmark results revealed a nearly 10x performance advantage for the Aerospike database running on a combination of native flash and DRAM over two other NoSQL databases running on a mix of DRAM and RAM: Apache Cassandra and 10gen MongoDB.

         Maximum Throughput – SSD-Backed Data Set

"Our goal with the study was to determine which key-value store is most appropriate in a real-world scenario. That meant not only using raw hardware and solid-state drives, but configuring each database to optimally handle the workload," said Ben Engber, Thumbtack CEO and co-author of the benchmark study. "Aerospike was the only vendor among those tested that specifically optimizes its platform around SSDs, so the Aerospike database’s superior performance was not unexpected."

Benchmark Overview
Thumbtack evaluated the three NoSQL database and key-value store offerings using the Yahoo Cloud Serving Benchmark (YCSB) as the basis for the testing. To gauge their capacity and speed in handling big data while operating against disk, Thumbtack tested the databases’ ability to process 500 million records. A fourth database, the Couchbase Server, was dropped from this evaluation because it could not load all of the records.

In the benchmark measuring a read-heavy workload, Aerospike achieved a maximum throughput of more than 300,000 operations per second, nearly 10x faster than its nearest competitor, Cassandra. For the balanced workload benchmark, Aerospike was 5x faster than Cassandra. In both tests, MongoDB placed third.

The Thumbtack report notes: "This was particularly impressive since both Cassandra and MongoDB were using weak consistency models (returning success after only one copy has been written) and Aerospike was using a strong consistency model."

Across all four Thumbtack tests for latency, Aerospike maintained sub-millisecond latency even while processing 180,000 to 400,000 operations per second. Latency for Cassandra ranged from sub-millisecond to 8 milliseconds, and for MongoDB, it varied from 1 to 20 milliseconds; meanwhile neither database exceeded 40,000 operations per second.

Flash-Based SSD Advantage
"From the start, we recognized that a database optimized for both flash and DRAM would be ideal for emerging classes of applications that live or die by their ability to reliably respond within milliseconds," said Brian Bulkowski, Aerospike founder and CTO. "Today our customers rely on our native flash Aerospike database to achieve both the real-time responses and 100% uptime that this extremely high performance enables. Now the ThumbTack benchmark adds significant new validation of the value in optimizing databases and applications for SSDs."

"Providing fast reliable access to data in real-time is simple to say, but it’s not easy to do," said Patrick DeAngelis, [x+1] CTO. "However, running our Aerospike clusters on SSDs, we’ve seen the database scale to a few billion key values with no compromise to performance, and we’ve even seen response times under 1 millisecond, which is phenomenal. We are now rethinking the way we store and access our data and intelligence because of the options that Aerospike gives us."

"We continue to push the boundaries of what you can do in a real-time environment, and Aerospike’s architecture, which is optimized for flash storage, means we can be a lot more efficient with the way we spread our data," said Mazdak Rezvani, Chango CTO. "Already we’ve seen a 25% improvement in efficiency with Aerospike over our previous NoSQL database."

Complementing Thumbtack’s NoSQL benchmark is the Aerospike Certification Tool (ACT) for benchmarking the ability of SSDs to support database transactions. ACT, which Aerospike has made available as open source software, can be downloaded.

To read the Thumbtack report, Ultra-High Performance NoSQL Benchmarking: Analyzing Durability and Performance Tradeoffs