Inkling-2-Mini in 5.4 GB Memory
Summary
The article discusses LookAtMyQuant, a quantization compiler, and its application to Inkling-2-Mini, showing dramatic model size reductions (5.38 GB and 8.87 GB) with low bits-per-weight. It covers installation, runtime on Apple Silicon, and benchmark results across multiple model configurations, highlighting open-source tooling and practical deployment steps.