Inverse Rubric Optimization: A testbed for agent science
Summary
The article presents inverse rubric optimization (IRO) as a testbed for agent science, examining how long-horizon agents learn judge preferences via prompts and label budgets. It reports on experiments across different rubrics and prompts (Milton, Shakespeare, Whitman, etc.), analyzes how budget and rubric design affect agent performance, and discusses open-source code and future work.