I built a programming language that LLMs can't use
Summary
The author introduces KillSwitch, an esoteric programming language designed to be challenging for LLMs and coding agents. It uses features such as hidden lies in the spec, letter-count constraints, and prompts intended to trigger guardrails, to test whether LLMs truly understand code or merely memorize patterns. Benchmark results are discussed, suggesting that while some models perform relatively well on simple tasks, this does not prove real comprehension, with the project openly available on GitHub and benchmarks at bench.killswitch-lang.org.