Roofline by Hand

Key Insight

Applying the roofline model by hand clarifies whether an operation is bounded by memory bandwidth or arithmetic throughput. By computing the arithmetic intensity of key steps like matmul, normalization, softmax, and GELU, we can predict which kernels will run slowly on a specific GPU (like an A100). This distinction prevents developers from wasting time optimizing FLOPS on operations that are actually bottlenecked by memory transfer.