Future of HBM Memory: From HBM4 to HBM8, The Era of Exabyte Bandwidth & Embedded Cooling

The future of computing is being built on the back of high-bandwidth memory (HBM), and the roadmap just got a massive update

. KAIST and Tera Labs have laid out the trajectory for HBM standards from HBM4 all the way to HBM8 – and it’s a wild ride filled with jaw-dropping bandwidths, memory capacities, and power demands that might make your GPU sweat just by reading this.

HBM4, launching in 2026, is at the forefront for next-gen AI GPUs. Both NVIDIA’s Rubin and AMD’s Instinct MI400 will use it. Rubin will feature 8 to 16 HBM4 stacks, up to 384 GB VRAM, and a total chip power nearing 2200W. AMD’s MI400 aims even higher with 432 GB capacity and nearly 20 TB/s bandwidth. The tech employs direct-to-chip liquid cooling and microbump packaging with each HBM consuming 75W.

HBM5, slated for 2029, takes things further with 4096 I/Os, 4 TB/s bandwidth per stack, and 80 GB per HBM site. NVIDIA’s upcoming “Feynman” GPU is expected to showcase this standard. Power jumps to 100W per HBM and up to 4400W per chip package, cooled using immersion techniques and featuring a decoupling capacitor chip and LPDDR+CXL enhancements.

HBM6 is where things get supercharged. Scheduled post-Feynman (~2032), it doubles the bandwidth to 8 TB/s and supports up to 120 GB per stack. Bump-less Cu-Cu bonding and immersion cooling are standard, while a hybrid silicon-glass interposer enables network switches within the memory architecture. One GPU package might reach 5920W and over 1900 GB of memory – a literal supercomputer on a chip.

HBM7 enters exabyte-class performance. With 8192 I/Os and 24 Gbps data rates, it offers up to 192 GB per stack and 24 TB/s bandwidth. This standard pushes twin-tower HBM designs with embedded cooling and a new HBM-HBF + LPDDR architecture. One massive package with 32 HBM7 stacks could offer 1024 TB/s bandwidth and over 6 TB of memory. Power draw? A staggering 15,360W.

HBM8 won’t land until around 2038, but it’s already a monster in the making. 32 Gbps speeds, 16,384 I/Os, and up to 240 GB per stack mean 64 TB/s bandwidth – per stack. It utilizes coaxial TSV, full-3D GPU-HBM integration, and double-sided interposers, with embedded cooling as a must-have. One setup could exceed 5000 GB memory and use enough power to rival a small data center.

Then comes HBF (High-Bandwidth Flash), a hybrid NAND-DRAM approach designed for AI/LLM inference. With 128-layer NAND in 16-Hi stacks and dedicated TSVs, each stack can add 1 TB capacity and integrate tightly with HBM via 2 TB/s links. Glass interposers and LPDDR up to 384 GB supplement the package – creating a unified, memory-centric architecture for the AI future.

From 2 TB/s to 64 TB/s per stack, and from gigabytes to terabytes of memory in one chip, HBM is no longer just a memory standard – it’s the foundation of the next generation of computing. The only question left is: can our power grids and cooling systems keep up?