SC25: d-Matrix and Andes Team on World’s Highest Performing, Most Efficient Accelerator for AI Inference at Scale

d-Matrix, a pioneer in generative AI inference compute for data centers, and Andes Technology, a supplier of high-efficiency, low power RISC-V processor cores and Founding Premier member of RISC-V International, today announced that d-Matrix has selected the AndesCore AX46MPV for its next-generation Raptor inference architecture.The collaboration represents a convergence of memory-centric computing and open-standard processor innovation for AI workloads at datacenter scale.

The Raptor architecture under development will be the first to feature d-Matrix’s revolutionary 3D In-Memory Compute (3DIMCTM) technology. A highly efficient chiplet-based approach and the world’s first 3D DRAM solution for AI inference at scale, 3DIMC stacks compute and memory to reduce the energy and latency of data movement by ten-fold, addressing the “memory wall” that limits conventional AI inference systems.

The Andes AX46MPV serves as both the orchestration engine and vector compute engine, managing workload distribution, memory coordination, and runtime control across Raptor’s high-bandwidth compute fabric as well as offloading computing tasks such as activation functions.

“Raptor’s innovative design is pushing inference efficiency and scalability far beyond what conventional, architectures can achieve,” said Sid Sheth, founder and CEO, d-Matrix. “The AX46MPV’s RISC-V architecture provides the ideal CPU for our 3DIMC approach, allowing us to deliver datacenter-scale inference with unprecedented performance per watt.”

“Our collaboration with d-Matrix showcases how RISC-V is enabling true datacenter-class inference systems,” said Frankwell Lin, chairman and CEO, Andes Technology. “By combining d-Matrix’s groundbreaking 3D In-Memory Compute with Andes’ advanced vector RISC-V CPUs, we’re bringing the next generation of generative AI inference to life.”

The partnership marks another milestone for RISC-V’s entry into large-scale AI inference, pairing Andes’ mature CPU IP and software ecosystem with d-Matrix’s innovative chiplet-based 3DIMC architecture. Together, the companies are redefining performance-per-watt and total cost of ownership (TCO) for generative AI deployment at scale.

Technology Highlights
The AX46MPV is a 64-bit, Linux-capable multicore CPU IP based on the open-standard RISC-V instruction set architecture. It includes up to 2048-bit vector processing unit (RVV 1.0) with performance boost in reduction sum, dual load-store unit for any combination of scalar/vector loads and stores, High-bandwidth Vector Memory (HVM) capable of handling out-of-order data returns, and a high degree of outstanding bus requests. Together, these features deliver the performance headroom and software flexibility required for datacenter-class inference systems. Reference kernels central to transformer and large-language-model workloads show up to 2.3x performance improvement compared to the processor’s predecessor, the AX45MPV.