Artificial Analysis v5 and the benchmarker's dilemma
Summary
This article analyzes Artificial Analysis v5's Intelligence Index and the benchmarking ecosystem, arguing that single-number indices struggle to capture real-world AI capabilities. It discusses the tension between open and closed models, private benchmarks, and the need for transparency and diversity in evaluation.