FreeToken: Edge-Native MoE Serving for Desktop Hardware
Summary
FreeToken is an edge-native Mixture-of-Experts (MoE) serving engine designed to run frontier-scale open-weight models on consumer hardware. It enables datacenter-scale model serving on desktops with features like bandwidth-adaptive execution, semantic-aware caching, and elastic memory management, while exposing Anthropic/OpenAI-compatible APIs. The project emphasizes a local, on-device workflow with desktop apps, CLI options, and extensive documentation.