Run Kimi K3 using 29 GB of RAM at 0.50 tok/s
Summary
WASTE is an embeddable C inference engine designed to run the full Kimi K3 model (2.78 trillion parameters) by streaming activated weights from disk on a consumer machine. The project demonstrates a 982 GiB container running on a 64 GB MacBook Pro with around 29 GB of RAM needed for 4K context and throughput around 0.49–0.54 tok/s, highlighting memory-centric design, MoE-based efficiency, and the practical limits of commodity hardware.