Three Reasons Why IT Leaders Should Get to Know CXL

This report was written on January 11, 2024 by Ken Clipperton, lead analyst, DCIG, LLC.

3 Reasons Why IT Leaders Should Get to Know CXL

Compute Express Link (CXL) is the new locus of data center innovation. It will soon prove to be a key enabler of improvements in data center performance and efficiency for hyperscalers and enterprises alike. David Hall, VP of Nvidia solutions at Lambda recently said, that the differentiation in data centers will be in the interconnect. CXL is that interconnect.

Significant advances in the CXL CPU-to-device interconnect were on display at SC23, and more are being revealed at CES. The CXL ecosystem includes IT infrastructure giants as well as many start-ups. At SC23, the CXL Consortium booth included demos from IntelliProp, MemVerge, Microchip, Micron, Samsung, XConn Technologies, and more. Though the products on display were, for the most part, engineering samples, these companies will begin to deliver the benefits of this rising interconnect standard to the marketplace beginning in 2024.

CXL Transforms PCIe to DCIE
PCIe stands for Peripheral Component Interconnect express. It is the ubiquitous standard for connecting high-speed components within a computer, including graphic cards and network interface cards. It uses the same hardware interface as PCIe but transforms it from a peripheral interconnect into a data center interconnect.

Beginning with CXL 3.0, CXL-enabled devices plugged into a PCIe interface are not mere peripherals. Instead, they become new nodes on a high-performance, low-latency data fabric. CXL was designed with shared memory in mind, so the protocols are very low overhead. Much more so than Ethernet, for example.

CXL enables data center performance
Prior to the emergence of NVMe, storage latencies were discussed in terms of milliseconds. As providers implemented end-to-end NVMe, milliseconds became microseconds. Multiple vendors now claim to deliver storage performance of less than 100 microseconds.

With CXL, the conversation shifts from microseconds to nanoseconds, the same latency category we use for DRAM. Early CXL memory products are demonstrating sub-200 nanosecond latencies. While nearly double the latency of a CPU’s DRAM, this is a dramatic improvement over any alternative approach.

Another way CXL enables data center performance is by making more memory bandwidth available to a CPU. My initial thoughts about the benefits of CXL memory were about how it could increase performance by providing more memory capacity to memory-intensive applications. However, one of the CXL demonstrations at SC23 showed CXL-attached memory delivering substantial performance improvements to an HPC application using the same amount of total memory as the same application running entirely on the same amount of local DRAM. The application had been memory bandwidth-bound, not memory capacity-bound.

CXL enables data center efficiency
Anyone paying attention to data center infrastructure is aware of the rise of the xPU. In addition to CPUs, we now have GPUs, NPUs, FPGAs, SmartNICs, and other accelerators. Up to now, these processors have been tied to a CPU in a specific appliance. With CXL, these devices become nodes on a fabric.

For example, Panmnesia recently announced an CXL-based AI accelerator that is claims speeds up AI searches by more than 100x. The achievement earned the company an Innovation Award at CES 2024. Another CXL innovator demonstrated a solution delivering 2x the performance while using 1/2 the power of traditional infrastructures.

CXL implications for IT infrastructure planning
While CXL is mostly at the engineering samples stage of delivery, multiple vendors tell me that they will move to production in 2024. They are seeing strong interest from hyper scalers due to the performance and efficiency that CXL can deliver. I believe that CXL-enable innovations will be available to enterprise technology architects within current technology refresh cycles for many organizations. Thus, enterprise IT leaders would do well to learn about CXL and track the progress of the technology in 2024 and beyond.