When is NVLink Worth It?
Summary
The article tests NVLink on two RTX 3090s to evaluate AI workloads, finding little benefit for inference with layer split or token generation but a near 3x uplift for FSDP training when PCIe bandwidth is limited. It also discusses Nvidia's consumer-GPU restrictions, patches to enable P2P, and the practical cost considerations, concluding that NVLink is worthwhile primarily for specific workloads and setups with constrained PCIe lanes.