Have it both ways: stay discoverable in search while disallowing AI training
Summary
Cloudflare announces a Disallow AI Training setting that lets site owners stay indexed in search while blocking AI training by crawlers. The post details Accountable crawlers from Apple, Google, and Microsoft, and introduces granular controls (Search, Training, Agent) applied at the domain level, with migration guidance for existing domains and plans for AI summaries opt-outs and future standards. It also outlines how these controls affect mixed-use crawlers and what site owners should do next.