Show HN: Run an 80B Qwen in 4.3 GB of RAM on a Mac, and a 35B on an iPhone
Summary
This GitHub project demonstrates running the 80B Qwen model in 4.3 GB of RAM on a Mac and 35B on an iPhone using Swiftlet, a Swift + Metal runtime for MoE-based transformers. It details the architecture (sparse MoE routing, qpack containers, on-device streaming), setup, and readiness status for end-to-end inference on consumer devices.