Giving a domain a hill to climb: benchmarking as data activation
Summary
The article frames benchmarking as a form of data activation in AI, describing how benchmarks turn domain data into measurable signals for model evaluation and training. It surveys multiple benchmark approaches (LatchBio, HealthBench, MedMarks, QuestBench) and discusses how running benchmarks as RL environments can make scoring and training tightly coupled, while highlighting challenges in high-stakes domains like medicine and the costs of data curation.