The 4-Bitter Lesson: Balancing Stability and Performance in NVFP4 RL
Summary
The 4-Bitter Lesson examines balancing stability and performance in NVFP4-based reinforcement learning. It introduces a baseline NVFP4 RL recipe, addresses gradient stability challenges, and presents four main techniques—Four-Over-Six quantization, dequantized backward, selective layer precisions, and an integrated final recipe—demonstrating improved gradient stability and deployment considerations. The article emphasizes open-source collaboration across TransformerEngine, FlashInfer, and related projects to achieve bit-exact, stable training and efficient serving.