BunnyCDN has been silently losing our production files for 15 months
Summary
The note treats each training example as a vector field and introduces Lie brackets to measure how swapping the order of examples affects parameter updates. It provides mathematical derivations and an experiment on a convnet trained on CelebA, including a PyTorch implementation for computing Lie brackets and observations about non-commutativity in training.