Running Gemma 4 26B at 5 tokens/sec on a 13-year-old Xeon with no GPU
Summary
A blog post detailing running Gemma 4 26B MoE on a 13-year-old Xeon with no GPU, achieving about five tokens per second decoding and about sixteen tokens per second for prompt evaluation. The author explains architectural constraints (AVX1 vs AVX2, lack of FMA), the mismatches in the ik_llama.cpp fork, and a patch workflow to enable CPU-only inference on pre-AVX2 hardware, including reproduction steps and a teardown of the bug.