MSA: Memory Sparse Attention – End-to-End Scalable Memory for 100M Tokens
Summary
MSA introduces an end-to-end trainable sparse attention framework with latent memory to extend context length to 100M tokens. It combines document-wise RoPE, KV cache compression, and a Memory Parallel inference engine to achieve high throughput, while a Memory Interleave enables multi-hop reasoning across memory segments; results show strong performance and stability across extremely long contexts compared to standard RAG setups.