On Violations of LLM Review Policies
Summary
ICML 2026 implemented two LLM review policies (A: no LLMs; B: LLMs allowed for understanding and polishing). The program flagged 795 reviews from 506 Policy A reviewers for LLM usage, leading to 497 desk rejections and removal of affected reviews; a watermarking-based detection method and a large phrase dictionary were used, with manual verification to avoid false positives. The post emphasizes protecting trust in peer review and notes potential disruption while outlining ongoing safeguards and policy refinements.