Running a 28.9M parameter LLM on an $8 microcontroller
Summary
The article showcases a 28.9 million parameter language model running entirely on an ESP32-S3 microcontroller, using Per-Layer Embeddings to store most parameters in flash. It discusses the hardware limitations, the memory strategy, and provides links to firmware, training code, and results, illustrating a highly edge-native approach to AI. This demonstrates a notable advance in on-device AI for constrained hardware and contains practical references for replication.