Pure Storage: STAC-M3 Benchmark Testing Results for High-performance and Quantitative Trading

By Diane Saucie, director, financial services, Pure Storage, Inc.

Pure Storage, Inc. has recently undergone benchmark testing with STAC (Strategic Technology Analysis Center LLC) to demonstrate the capabilities of the FlashBlade//S500 series. Here are the results.

In the fast-paced world of high-frequency and quantitative trading, every microsecond counts. Financial firms are constantly on the lookout for data platforms that offer better performance, scalability, and reliability for their quant analysts, data scientists, risk managers, and others supporting their trading teams. 

The company, provider in cloud-ready solutions, has recently submitted a solution for benchmark testing by STAC to demonstrate the capabilities of our FlashBlade//S500 series.

This blog post unveils the results and showcases the firm’s commitment to delivering best-in-class solutions for the financial sector.

Benchmark testing overview
STAC recently conducted baseline and scaling STAC-M3 Benchmarks on a comprehensive stack that involved Pure’s FlashBlade//S500 ⁽¹⁾. The stack featured KX’s kdb+ 4.0 DBMS, utilizing NFS version 3 to access a 266TB of total usable data across a single FlashBlade//S500 chassis (10xFlashBlades with 2x24TB flash modules per blade). The kdb+ software was distributed across 8xDell PowerEdge R740xd servers, each equipped with 2xXeon Platinum 8260 CPUs and 256GB of memory.

The STAC-M3 Benchmark suite is an industry standard for tick-analytics benchmarks, designed for database software/hardware stacks managing large time series of market data. The results from both the baseline (Antuco) and scaling (Kanaga) suites demonstrate Pure’s commitment to delivering performance.

This is the 1st solution of the company has submitted for a STAC-M3 audit. Many participants tune their infrastructure stack and OS extensively to get the best benchmarks possible. In contrast, the manufacturer chose to have the benchmark harness run with a baseline FlashBlade//S500, a common server and network architecture, and minimal configuration tuning to demonstrate that its customers can achieve maximum performance and simplicity when choosing the firm as their time-series or tick-by-tick market data platform.

STAC-M3 and Pure audit results

Faster performance
The company’s solution involving kdb+ 4.0, Compatibility Rev I of the kdb+ STAC Pack, and 8 database servers accessing a FlashBlade//S500 via NFSv3 showcased remarkable speed gains vs. a solution involving the same number of database servers, 3 network attached flash storage nodes, and Compatibility Rev H of the STAC Pack ⁽²⁾:

• 13 of 17 STAC-M3 Antuco mean-response time benchmarks witnessed improvements, including a 7x speedup in the 10-user theoretical P&L (STAC-M3.β1.10T.THEOPL.TIME).

• 17 of 24 STAC-M3 Kanaga mean-response time benchmarks exhibited enhancements, such as a 1.3x-1.5x speedup in 50-user 12-day VWAB (STAC-M3.β1.50T.YR{1,2,3,4,5}VWAB-12D-HO.TIME).

Additionally, when compared to a cloud-based solution ⁽³⁾, the FlashBlade//S500 outperformed in 9 of 17 Antuco benchmarks and 12 of 24 Kanaga benchmarks, showcasing significant speed advantages.

FlashBlade//S appliance

Scalability
These results validate the FlashBlade//S500 series’ performance for financial sector tick data and time series-driven workloads. The platform enables firms to run complex queries across data sets of any scale with maximum throughput and minimum latency.

As workloads increased in volume and thread count, FlashBlade//S500 demonstrated the ability to handle highly parallel workloads efficiently. The architecture allowed for seamless scalability, crucial for the demanding requirements of quant and high-frequency trading.

Jack Gidding, CEO, STAC, said: “We are pleased to welcome Pure Storage to the ranks of technology providers who submit solutions for independent testing against the customer-developed STAC Benchmark standards. Financial firms on the STAC Benchmark Council designed STAC-M3 to test a representative range of business use cases, and the need for fast and efficient time-series analysis has never been higher.”

STAC-M3 Antuco results were obtained with the entire data set centralized in a single directory, showcasing the efficiency that firms can gain by eliminating the necessity to distribute data across multiple, specific directories for optimal performance. This approach underscores the streamlined and effective data management facilitated by Pure Storage’s solution.

FlashBlade//S500: Performance for quantitative and high-performance trading
The benchmarked system featured FlashBlade//S500, a purpose-built, all-QLC flash, unified fast file and object (UFFO) storage platform. FlashBlade//S is designed to deliver performance, throughput, and continuous uptime – critical requirements for data scientists and quantitative analysts running multi-threaded, time-series workloads.

Key features of FlashBlade//S500 include:

• Performance and throughput: Equipped with advanced version of the Purity//FB software, it ensures maximum performance for analysis, backtesting, and execution in the world of quant and high-frequency trading.
• Modular architecture: The modular architecture enables independent scaling of capacity and performance. This flexibility allows organizations to efficiently scale their infrastructure to meet the demands of varying workloads.
• Unified platform: It combines compute, storage, and networking onto a single unified platform, simplifying deployment, management, and upgrades.
• Native multi-protocol access: Supporting NFS, S3 object, and SMB in a single system, it provides native multi-protocol access, enhancing its versatility.
• Space, power, and cooling efficiency: Consuming only 5 rack units initially, the disaggregated architecture scales out capacity and performance linearly to multiple chassis, offering industry-leading space, power, and cooling efficiency.
• Pure Storage DirectFlash Modules: The firm differentiates itself with DirectFlash Modules, eliminating the need for SSDs or HDs in arrays, ensuring performance, reliability, expandability, security, and simplicity.

Solutions optimize trading
In addition to FlashBlade//S, the company offers many solutions to meet the intensive data demands of high-frequency trading.

These solutions include: 

• FlashBlade//E: Economics without compromise
  It provides all the benefits of all-flash with better economics than HDD. It’s a capacity-optimized unified file and object storage platform designed to optimize massive unstructured data growth. It offers optimal energy and capacity efficiency at multi-petabyte scale, making it for repository workloads required for quant trading, data protection, AI/ML, and more.
• FlashArray//XL: Scale up and scale out with native NVMe
  It offers 100% native NVMe storage optimized for performance at scale. With performance density, it allows organizations to consolidate more business services – bigger databases, more applications, more users – on fewer arrays.
• Portworx: Kubernetes data platform
  It is a Kubernetes data platform for building, automating, protecting, and securing cloud-native applications. It enables the management of mission-critical applications across any cloud and any infrastructure with zero downtime or data loss while lowering infrastructure costs. 
• Evergreen: Future-proof your infrastructure
  It provides a non-disruptive extensible architecture for upgrades and expansions, eliminating worries about service interruptions, functionality decreasing over time, or expensive and risky data and hardware migrations. It ensures organizations can adopt the best and newest technology while the Evergreen as-a-service portfolio offers unequaled flexibility in purchasing and consuming data storage.

Conclusion
In the competitive landscape of high-frequency and quant trading, the FlashBlade//S500 series stands out as a reliable and high-performance solution. The STAC-M3 benchmark testing results demonstrate the platform’s capability to deliver faster performance and scalability, addressing the needs of the financial sector.

Trading firms and financial institutions seeking data solutions should explore the detailed benchmark results in the STAC Report and visit the Pure Storage quant trading web page for a  understanding of the innovative capabilities offered by the company’s all-flash solutions.

⁽¹⁾ STAC Report: kdb+ 4.0 with Pure FlashBlade//S500 under STAC-M3 (tick analytics)    
⁽²⁾ kdb+ 4.0 on 8 x Dell PowerEdge R7525 servers accessing a 3-node Dell PowerScale F900 All-Flash cluster    
⁽³⁾ kdb+ 4.0 on 15 x Amazon EC2 C5n.9xlarge instances with Intel® Xeon Platinum 8124 CPU @ 3.0GHz and WekaIO WekaFS v3.10.1 Parallel Filesystem on 40xAmazon EC2 i3en.6xlarge instances