Open-Sourcing Sarvam 30B and 105B: India's First Competitive Open-Source LLMs
Summary
Sarvam releases 30B and 105B open-source LLMs built in India, detailing a full-stack sovereign AI stack from data curation to deployment. The post highlights a Mixture-of-Experts Transformer with features like Grouped Query Attention and Multi-head Latent Attention, strong Indian-language benchmarks, and extensive inference optimizations for diverse hardware, with access via API and downloadable weights.