SC23: Xinnor xiRAID Breaking Record Performance for PCIe Gen5 SSDs

In August, at Flash Memory Summit, Xinnor demonstrated world record performance with its xiRAID and 12 Kioxia Corp. CM7-R Series PCIe 5.0 NVMe read-intensive SSD.

Click to enlarge

This time, Kioxia provided for testing 24 units CM7-V Series PCIe 5.0 NVMe mixed-use SSD. Each drive is rated 14GB/s sequential read and 6.75GB/ sequential write, with random read being rated at 2.7 million IO/s and random write over 600,000 IO/s. These drives are for many applications, including HPC and AI.

When deploying several of these drives in a system, assuring proper performance scalability and data integrity becomes paramount.

The company performed the test on a Supermicro server with 2 AMD Genoa CPU with 64 Core each. To avoid intra Numa nodes performance bottlenecks, the company created 2xiRAID groups with RAID-5 over 12 drives, each group connected to a single NUMA node. This configuration provides best performance and at the same time, data protection in case of drive failure for each RAID group.

Traditionally, HPC installations have been accessing data sequentially. The emergence of AI workload is creating more challenges to the underlying storage as the access is a mix of random and sequential patterns.

As shown in this table, results achieved by xiRAID are impressive in every workload ^(*).

Video that shows dashboards with system performance for random/sequential writes/reads

Alternative software and GPU-assisted RAID implementations trade performance with CPU load. This is not the case for xiRAID: in sequential workloads, it’s remarkable that these record performances were achieved by using only 16 out of the 128 available cores for running both the data flow as well as xiRAID calculations. In random operations, xiRAID calculations only required 3-9% of half of the cores. Such low CPU utilization is critical in compute intensive application as all the other available cores can be used for what they are meant for: running the HPC/AI models.

The company also tested what happens in degraded mode. Degraded mode in RAID-5 is when 1 drive is taken out of the RAID group. Indeed, being able to keep operating without any data loss and maintain good performance in degraded mode is the main reason why RAID is required. When emulating degraded mode, the performance remained high, with random RW respectively exceeding 34 million and 6 million IO/s. This is equivalent to 52% and 75% of the performance in normal operation. In sequential workload, the firm measured performance above 70% of normal operations. These percentages approach the theoretical maximum, demonstrating once again the superiority of xiRAID architecture.

In addition to RAID-5, the vendor also tested the same system by creating 2 groups of RAID-6 (2×10 drives for data and 2 drives for parity). This configuration can withstand the failure of 2 drives for each RAID group.

In read workload, the performance in RAID-6 were almost the same as measured in RAID-5: 64 million IO/s in random and 310GB/s in sequential. In write operation, because of the need to calculate and write one additional parity, the performance was slightly lower (5.2 million IO/s in random and 128GB/s in sequential) but still above anything demonstrated so far by any alternative RAID solution.

The company will showcase the results of this test at SC23 in Denver. CO.

^(*) Testing system configuration: xiRAID configuration: 2xRAID-5; Server: 1xSupermicro AS-2125HS-TNR, CPUs: 2xAMD EPYC 9534 64-Core Processor @ 2,450MHz, Memory: 24xMicron 32GB DIMMS @ 4,800MT/s. Drives: 1xKioxia CD7 (boot drive), 24xKioxia CM7 3.2TB KCMYXVUG3T20 drives, OS: Ubuntu 22.04.3 LTS, Kernel: 5.15.0-87-generic