NVIDIA Model Optimizer
Summary
NVIDIA Model Optimizer is an open-source library that provides state-of-the-art model optimization techniques (quantization, pruning, neural architecture search, distillation, speculative decoding, and sparsity) to accelerate AI model inference. It supports inputs from Hugging Face, PyTorch, and ONNX, and integrates with Megatron-Bridge, Megatron-LM, and Hugging Face Accelerate for training-aware optimization; outputs can be deployed to TensorRT-LLM, TensorRT, vLLM, and SGLang. The project maintains ongoing news, roadmaps, benchmarks, and pre-quantized checkpoints with an active open-source collaboration model.