Micron Warns: AI Compute Outpaces HBM
According to Micron, the rapid growth of AI computing power is widening the gap between processing performance and memory capability, making the “memory wall” an increasingly critical bottleneck for AI systems.
At Hot Chips 2026, held at Stanford University on August 23, Raghu Sreeramaneni, a Micron HBM architecture researcher, said computing capacity is growing by roughly three times every two years, while memory capability increases by less than two times over the same period. This widening gap means processors are increasingly forced to wait for data.
High-bandwidth memory (HBM) has become a key DRAM technology for advanced AI accelerators because of its significantly higher bandwidth and parallelism compared with conventional DDR5. A typical HBM system can deliver up to 5.3 TB/s of system-level bandwidth, far exceeding the roughly 300 GB/s typical of DDR5 configurations.
However, higher performance comes with greater complexity. Micron noted that in a typical GPU system-in-package using four 12-high HBM stacks, memory silicon can account for around 90% of the total silicon area, or about eight times the GPU silicon area. As HBM stacks become taller and denser, thermal and mechanical challenges are also becoming more severe.
Micron is therefore exploring technologies beyond simply increasing HBM bandwidth and stack height. Its HBM4 solution doubles the number of channels and I/O speed compared with HBM3E, with system bandwidth reaching up to 2.8 TB/s and up to 2,000 I/O connections.
To address rising power density and heat, Micron is also developing advanced solutions including liquid cooling, hybrid bonding and fusion bonding. These technologies could reduce thermal resistance while enabling tighter interconnections and higher data-transfer density.
As AI models continue to expand, overcoming the memory wall will require advances across memory architecture, high-speed I/O, advanced packaging and thermal management, rather than relying on computing performance alone.
