Introducing The Conceptual Reasoning Index
Summary
Anthropic and Redwood Research introduce the Conceptual Reasoning Index (CRI), a composite benchmark suite combining LMCA, ACCoRD, and DTBench to measure models' conceptual reasoning capabilities relevant to AI risk mitigation. The post outlines methodology, benchmarks, and current results, noting that AI safety work often involves reasoning beyond easily verifiable feedback and that CRI aims to track progress and guide future benchmarking. It also highlights ongoing work and open access to datasets and CRI live scores.