Asymmetric Quantization: Near-Lossless Late interaction Retrieval with 97% Storage Reduction
Summary
The article introduces asymmetric quantization for late interaction retrieval, achieving near 97% storage reduction by storing document vectors as binary signs while keeping query vectors at higher precision. It evaluates retrieval quality with NDCG@10 and shows a minimal drop despite dramatic storage gains, enabling scalable vector search at billions of documents. It also details the scoring trick and practical implications for production systems using silo-like architectures.