The Most Forbidden Technique
Summary
The article explores 'The Most Forbidden Technique'—training AI on interpretability outputs and the risks of optimization pressure on chain-of-thought (CoT) monitoring. It surveys OpenAI's findings on reward hacking, CoT monitorability, and the tension between safety-oriented prompting and model performance, with broader implications for AI safety research and policy.