Speedb Goes Open Source with Data Engine Drop-in Replacement for RocksDB

Speedb Ltd., the company behind the Speedb data engine, a drop-in replacement for RocksDB, is going open source with the technology it has been delivering in its enterprise edition for the past 2 years.

The company’s open source data engine brings an opportunity for the development community to take part in shaping the data architecture of the future. The firm’s open source community serves as a hub for Speedb and RocksDB users and contributors to collaborate, and share knowledge, issues and solutions.

Speedb is sharing key features from its enterprise version to the open source community to support the development of capabilities that are becoming essential in addressing the needs of modern, data-intensive workloads.

Open sourced by Facebook as a fork of Google’s LevelDB in 2012, RocksDB is the de facto standard embedded key value store (KVS). As RocksDB adoption expands to cover a broader range of use cases and deployment scenarios, the Speedb community fills a gap in the KVS space by providing developers with innovation, support and services to overcome emerging challenges.

RocksDB is everywhere, yet it struggles with scaling, and thus requires sharding and performance-tuning that consume undue developer time as applications grow. Speedb redesigned key parts of RocksDB including a compaction algorithm that reduces the Write Amplification Factor (WAF) from ~30 to ~5 – as well as redesigning its I/O and flow mechanisms.

Based on these capabilities, the data engine will be working with the open source community to address issues such as the explosion of metadata, which is increasingly impacting the performance of databases, streaming applications, AI/ML, cybersecurity, analytics, and more. By changing a few lines of code, RocksDB users can embed the Speedb data engine into their apps to boost metadata memory performance, and eliminate the need to constantly deal with time-consuming operational tasks, so they can focus on delivering real business value.

With its open source edition, Speedb is poised to capture demand for new database management and development tools that are suitable for modern applications. When developers’ applications grow in size and complexity, Speedb’s enterprise edition is a ready option for additional features, higher scalability, support services and tuning.

Speedb’s open source edition is available at GitHub and Speedb’s dev site.

“We’ve been using Speedb internally for a while now, and we’re very excited for the promise it shows over RocksDB, in memory management, compaction, and configurability,” said Haikal Pribadi, CEO, Vaticle, creators of the strongly-typed database, TypeDB, and its query language, TypeQL. “Most importantly, having the dedicated support of Speedb in our development efforts on top of the storage infrastructure has been invaluable. Speedb already has made a huge difference to our team.”

The Speedb enterprise edition enables users to routinely generate up to double throughput, with half the latency, and double the capacity as compared to the same deployment with the RocksDB storage engine.

“There’s a new wave of application data native development solutions that promise to significantly cut down on development time, infrastructure hassles, and overall technical debt for data-intensive workloads,” said Brad Shimmin, chief analyst, AI platforms, analytics and data management, Omdia. “This can serve as a significant advantage for overburdened IT departments, seeking to make good on high-profile, demanding modernization and transformation projects. Speedb is clearly targeting an area of strong demand where innovations address a highly receptive community.”

“By going open source, Speedb can help far more developers to remove the key barriers to data performance, and take their hyperscale data operations to the next level, backed by a supportive developer community,” said Adi Gelvan, CEO and co-founder, Speedb. “We’ll be making ongoing additions of our enterprise functionality to the open source edition, and looking forward to embracing the community assistance and collaborating with the many creative minds in the developer realm that are also searching for more efficient ways to rev their data engines.”

Resources:
Blog : How does RocksDB Memory Management work?    
Blog : Understanding Leveled Compaction    
Blog : LSM vs B-Tree    
Blog : What Factors Affect Performance in RocksDB?