NCCL Tests

Key Insight

Running NVIDIA's official NCCL (NVIDIA Collective Communications Library) benchmarks allows developers to isolate and measure raw AllReduce and AllGather bandwidth across multiple GPUs. By bypassing training frameworks and model compute overhead, these tests establish a clean hardware baseline for communication latency and throughput over NVLink or InfiniBand (IB) networks. This isolation is crucial for diagnosing network hardware issues, driver bottlenecks, or topology misconfigurations before launching expensive, large-scale training jobs.