Micron HBM3E 12-High Boasts 36GB Capacity

Written by Raj Narasimhan, SVP and GM, compute and networking business unit, Micron Technology, Inc.

As AI workloads continue to evolve and expand, memory bandwidth and capacity are increasingly critical for system performance.

The latest GPUs in the industry need the highest performance high bandwidth memory (HBM), significant memory capacity, as well as improved power efficiency. Micron is at the forefront of memory innovation to meet these needs and is shipping production-capable HBM3E 12-high to key industry partners for qualification across the AI ecosystem.

HBM3E 12-high boasts a 36GB capacity, a 50% increase over current HBM3E 8-high offerings, allowing larger AI models like Llama 2 with 70 billion parameters to run on a single processor. This capacity increase allows faster time to insight by avoiding CPU offload and GPU-GPU communication delays.

HBM3E 12-high 36GB  delivers lower power consumption than the competitors’ HBM3E 8-high 24GB  solutions. It offers more than 1.2TB/s of memory bandwidth at a pin speed greater than 9.2Gb/s. These combined advantages of HBM3E offer maximum throughput with the lowest power consumption can ensure optimal outcomes for power-hungry data centers.

Additionally, HBM3E 12-high incorporates programmable MBIST that can run system representative traffic at full spec speed, providing improved test coverage for expedited validation and enabling faster time to market and enhancing system reliability.

Robust ecosystem support
Micron is shipping production-capable HBM3E 12-high units to key industry partners for qualification across the AI ecosystem. This HBM3E 12-high milestone demonstrates firm’s innovations to meet the data-intensive demands of the evolving AI infrastructure.

The vendor is also a partner in TSMC’s 3DFabric Alliance, which helps shape the future of semiconductor and system innovations. AI system manufacturing is complex, and HBM3E integration requires close collaboration between memory suppliers, customers and outsourced semiconductor assembly and test (OSAT) players.

In a recent exchange, Dan Kochpatcharin, head of the ecosystem and alliance management division, TSMC, commented: “TSMC and Micron have enjoyed a long-term strategic partnership. As part of the OIP ecosystem, we have worked closely to enable Micron’s HBM3E-based system and chip-on-wafer-on-substrate (CoWoS) packaging design to support our customer’s AI innovation.“

In summary, here are the HBM3E 12-high 36GB  highlights:

• Undergoing multiple customer qualifications: The firm is shipping production-capable 12-high units to key industry partners to enable qualifications across the AI ecosystem.
• Salability: With 36GB of capacity (a 50% increase in capacity over current HBM3E offerings), HBM3E 12-high allows data centers to scale their increasing AI workloads.
• Exceptional efficiency: It delivers lower power consumption than the competitive HBM3E 8-high 24GB  solution!
• Superior performance: With pin speed greater than 9.2Gb/s, it  delivers more than 1.2TB/s of memory bandwidth, enabling lightning-fast data access for AI accelerators, HPCs and data centers.
• Expedited validation: Fully programmable MBIST capabilities can run at speeds representative of system traffic, providing improved test coverage for expedited validation, enabling faster time to market and enhancing system reliability.

Looking ahead
Micron’s data center memory and storage portfolio is designed to meet the evolving demands of generative AI workloads. From near memory (HBM) and main memory (high-capacity server RDIMMs) to Gen5 PCIe NVMe SSDs and data lake SSDs, the company offers products that scale AI workloads.

As the firm continues to focus on extending its industry leadership, it is already looking toward the future with its HBM4 and HBM4E roadmap. This forward-thinking approach ensures that it remains at the forefront of memory and storage development, driving the next wave of advancements in data center technology.

More information in the HBM3E page