HBM Saturation

Key Insight

Saturating the off-chip HBM bandwidth is the defining limit for memory-bound kernels on a GPU. In simple operations like vector addition, the low arithmetic intensity keeps the compute units idle while the hardware waits for data. Maximizing the achieved memory bandwidth on the roofline diagram requires tuning instruction-level parallelism and load sizes to keep the memory controllers fully utilized.