Introducing FrontierHarness Eval v1.1
Summary
This post introduces FrontierHarness Eval v1.1, a large-scale benchmark that compares 22 harnesses across 5 models over 62 tasks, totaling 27,280 agent runs and 83B tokens. It demonstrates that harness choice significantly impacts cost and performance, with the model largely driving cost and 64.5% of spend going to trials that did not pass. The article provides methodology, model-harness pairings, and recommendations to test harnesses with your model and dataset to optimize value.