Debugging performance regressions
Summary
This article documents debugging a performance regression in an MPI stack used in high-performance computing with Guix. It explains how a regression on the Slingshot interconnect caused peak bandwidth to drop, and demonstrates a reproducible workflow using guix time-machine and commit-level bisecting to identify the offending libcxi upgrade. The piece highlights the benefits of reproducible toolchains for performance troubleshooting in HPC environments.