WekaIO Unveils Performance on STAC-M3 Tick Analytics Benchmark

WekaIO, Inc. announced record performance on the STAC-M3 Benchmark from an integrated solution with Penguin Computing FrostByte storage solution featuring its File System.

Click to enlarge

The STAC-M3 Antuco and Kanaga benchmark suites are the industry standard for testing solutions that enable high-speed analytics on time series data, such as tick database stacks.

The company’s File System on Penguin Computing Relion servers and FrostByte storage system delivered record performance at scale for the STAC-M3 ‘Tick Analytics’ benchmark, affirming that the integrated solution is the choice for algorithmic trading and quantitative analysis workloads common in financial services.

The STAC-M3 benchmark specifications are maintained by the STAC Benchmark Council, which consists of over 300 financial institutions and more than 50 vendor organizations whose purpose is to discuss technical challenges and solutions in financial services and to develop technology benchmark standards that are useful to financial organizations. User firms include the largest global banks, brokerage houses, exchanges, hedge funds, proprietary trading shops, and other market participants. The STAC-M3 results were audited by the Securities Technology Analysis Center (STAC), which facilitates the Council.

The system under test was a Kx kdb+ 3.6 database system distributed across 7 Penguin Computing Relion XO1132g servers with dual Xeon Gold 6140 CPU @2.30GHz and data stored on FrostByte with the firm’s File System (v3.2.2) Software running on eight 1U RXE1112 server nodes with nine NVMe SSDs per server, using Mellanox SB7780 36-port Non-blocking Managed EDR 100Gb/s IB Router and IB EDR 100GbE R 1-port 840QSFP28 adapters.

The tests were run on the baseline benchmark suite (Antuco) and the scaling suite (Kanaga), a test suite with 5 years of simulated stock market data. The test system had a combined performance density of up to 87.5GB/s available bandwidth to the client nodes, 40.5GB/s of actual throughput ^(*), and 2.5 million 4K IO/s in 8U, with the ability to distribute data, metadata and system services, as well as low file system latency that leverages high-speed networks and NVMe-optimized storage.

The test system delivered record performance, breaking 8 STAC-M3 world records for mean query-response times and 4 world records for throughput.

Kx’s time-series database kdb+ is commonly used for large scale complex analytics on streaming, real time, and historical datasets in industries ranging from financial services to marketing analytics, and for Industrial IoT. In the financial industry, typical kdb+ applications are for algorithmic trading, back testing (trading algorithm validation), surveillance, regulatory and compliance reporting and research environments. These workloads can demand a lot from the file system.

Better than block-based solutions for financial services, the company’s File System is the only NVMe-optimized, shared, coherent, POSIX-compliant file system with a distributed architecture for data resiliency. The firm’s software eliminates islands of storage, providing all applications access to a shared data pool with record performance. The File System delivers 10x better performance than a local file system by bending the rules on data locality, distributing the data across the cluster over low latency IB or Ethernet networks.

The company is a firm believer in the value of benchmark testing as a means to guide and inform development in the industry and frame results in a manner that delivers actionable intelligence to customers.

The firm’s File System currently holds one of the top positions on the independent SPEC SFS 2014 benchmark and the #1 position on VI4IO’s 10 Node Challenge list.

STAC report (registration required) contains an explanation of the benchmark specifications and the results for this solution

“Traditionally, IT organizations have used block-based storage solutions for their trading databases, rather than compromise performance due to network and file system latencies. The result has been large investments in specialized hardware and software to run different applications,” said David Hiatt, director, business development and alliances, WekaIO. “Now with the STAC-M3 Benchmark results, we have proven that with the WekaIO File System on Penguin Computing Relion servers and FrostByte delivers better application performance for Kx kdb+ by running in distributed mode over a network than with traditional block storage, with the additional benefit of simultaneously supporting other applications on the same platform. That’s a huge leap in application acceleration and simplifying the storage infrastructure for financial services.“

“The Penguin Computing FrostByte integrated solution featuring the WekaIO File System is designed, engineered and vetted to deliver a high-performance storage solution for Kx kdb+,” added Dr. Kevin Tubbs, senior director technology, advanced solutions group, Penguin. “To break eight STAC-M3 performance records is an incredible achievement and a testament to the innovative architecture of the WekaIO File System and leading HPC and AI solutions from Penguin Computing. For IT supporting different applications and Kx kdb+ time-series databases, this platform is an unbeatable combination for performance, scale, and ease of management.“

Glenn Wright, systems architecture expert, Kx Systems, Inc., said: “Kx customers demand both high performance and low latency for compute and storage. The ability to combine both a distributed kdb+ implementation running against a high-performance shared file system can be attractive to our customers. With WekaIO as this file system, our customers are able to choose how and when they distribute their historical data workload to be accessible by one or more of their analytics workloads.“

“It’s great to see another successful benchmark test outcome when the WekaIO File System is paired with Mellanox’s network adapters. We have successfully collaborated with WekaIO on many benchmarks and together, we continue to set benchmark records,” contributed Gilad Shainer, SVP marketing, Mellanox Technologies Ltd. “The Mellanox IB EDR/100GbE 1-port 840QSFP28 adapters provide the lowest latency available, and that coupled with the WekaIO distributed and parallel file system satisfies the performance of even the most demanding I/O intensive application requirements.“

“There’s now a proliferation of software and hardware configurations aiming to support I/O intensive workloads,” commented David Floyer, co-founder and CTO, Wikibon. “But to meet the performance demands of real-time and historical analytics in kdb+ environments, storage systems must offer both high performance and low latency. The results from the STAC-M3 Benchmark should help IT in making an informed decision when architecting storage systems to support algorithmic trading and quantitative analytics workloads.“

^(*) As per the benchmark STAC-M3.ß1.1T.5YRHIBID.MAX