Taming LLMs: Using Executable Oracles to Prevent Bad Code
Summary
The article advocates giving AI code-writing agents zero degrees of freedom by using executable oracles (like test suites, fuzzers, and verifiers) to prevent bad code. It discusses concrete examples, practical tooling considerations, and the importance of robust metrics to avoid gaming the system, aiming for reliable, testable AI-assisted software development.