Huawei releases MACaQ, its internal quantizer for local inference
Summary
MACaQ is a model-agnostic quantization tool for GGUF-based local inference. It supports quantizing and running text, image, video, and audio models across backends like llama.cpp, vLLM, Diffusers, and PyTorch, claiming fast performance (e.g., quantizing a 122B-parameter model in 64 minutes using 32.5 GB) and improved perplexity versus MR-GPTQ and DuQuant++. The project is open-source, with backend-specific guidance and an Apache 2.0 license.