Why your local LLM feels dumber than it is
Summary
The post argues that local LLM performance is highly dependent on hardware and software stacks, quantization, and inference backends. It explains core concepts like logits, KL divergence, and attention backends, and compares several quantization approaches across Qwen 3.6-27B, highlighting tool-call failures due to quantization. The author stresses rigorous benchmarking and reproducibility, with plans for deeper future experiments.