Machine Learning
Maarten Grootendorst presents a visual, accessible guide to quantization for LLMs, covering FP32/FP16/BF16, INT8, symmetric and asymmetric quantization, and both post-training and quantization-aware training. The piece also explores advanced topics like GPTQ and GGUF, 1-bit BitNet and 1.58-bit quantization, including memory and accuracy trade-offs with diagrams and examples.