Oh My Algorithm
Concept Guide3D-Stacked · High Bandwidth

HBM (High Bandwidth Memory)

HBM is a high-bandwidth memory that stacks several DRAM dies vertically, connects them with TSVs (through-silicon vias), and places them right beside the GPU on a silicon interposer over a 1024-bit ultra-wide bus. In a small footprint it achieves large capacity, high bandwidth, and low power at once.

01HBM (High Bandwidth Memory)

Legacy · GDDR (before HBM)

Before HBM, GDDR was used. DRAM chips are placed separately on the PCB beside the GPU.

Each chip connects to the GPU over long traces on the PCB. A single chip's bus is narrow (e.g. 32-bit), so speed comes from pushing the clock much higher instead.

Long distances and a narrow bus make bandwidth hard to scale, and the high clock draws heavy power and heat. These limits are exactly why HBM emerged.

GDDRDRAM
GDDRDRAM
GPU / ProcessorMemory Controller
GDDRDRAM
GDDRDRAM
PCB board · long traces · 32-bit narrow bus × many
101
1 / 3

HBM · 3D-Stacked High-Bandwidth Memory

HBM takes the opposite approach to GDDR. Instead of spreading DRAM sideways, it stacks it vertically right beside the GPU.

Several DRAM dies are stacked upward. Without spreading out flat, a small footprint holds large capacity.

At the bottom of the stack, the base logic die handles the PHY (physical interface) and control circuits, linking the DRAM dies above to the outside.

TSVs (through-silicon vias) pass straight up through the dies to connect them. Unlike GDDR's long PCB traces, the paths are short and number in the thousands.

The HBM stack and GPU sit side by side on a silicon interposer, directly connected by a 1024-bit ultra-wide bus — in contrast to GDDR's narrow 32-bit bus.

Thanks to the wide bus and short distance, even at a low clock it delivers far higher bandwidth than GDDR, with better power efficiency too.

HBM stack
DRAM Die
DRAM Die
DRAM Die
DRAM Die
Base Logic DiePHY · Control
101
101
TSV
GPU / ProcessorMemory Controller
Silicon Interposer · 1024-bit wide bus
10110110DATA100101011100
Package Substrate
1 / 6

In short

vertical stacking (capacity) + TSVs (short, plentiful paths) + interposer wide bus (bandwidth). It solves GDDR's limits structurally, powering AI accelerators and HPC GPUs.

02 Understand It Simply

For Everyone
🔑How It Works

Memory dies are stacked vertically instead of spread sideways, joined by thousands of TSV channels. The stack sits beside the GPU on an interposer with a very wide bus, giving large bandwidth at a low clock.

💡In Plain Words

It stacks DRAM vertically to save area and uses TSVs to connect dies straight through, making paths short and plentiful.

It sits beside the GPU on an interposer over a 1024-bit-wide bus, so even at a low clock it moves wide data at once — higher bandwidth and better power efficiency than GDDR.

📍Where It's Used
  • –Memory for AI accelerators
  • –HPC GPUs
  • –and high-performance network gear; bandwidth-bound workloads; understanding the trade-off against GDDR

03 Frequently Asked Questions

FAQ
What is HBM (High Bandwidth Memory)?+

HBM is a high-bandwidth memory that stacks several DRAM dies vertically, connects them with TSVs (through-silicon vias), and places them right beside the GPU on a silicon interposer over a 1024-bit ultra-wide bus. In a small footprint it achieves large capacity, high bandwidth, and low power at once.

Where is HBM (High Bandwidth Memory) used?+

Memory for AI accelerators, HPC GPUs, and high-performance network gear; bandwidth-bound workloads; understanding the trade-off against GDDR.

What's a simple analogy for HBM (High Bandwidth Memory)?+

Memory dies are stacked vertically instead of spread sideways, joined by thousands of TSV channels. The stack sits beside the GPU on an interposer with a very wide bus, giving large bandwidth at a low clock.