DigiNews

Tech Watch by Johan Denoyer

← Back to articles

Introducing FrontierHarness Eval v1.1

Quality: 8/10 Relevance: 9/10

Summary

This post introduces FrontierHarness Eval v1.1, a large-scale benchmark that compares 22 harnesses across 5 models over 62 tasks, totaling 27,280 agent runs and 83B tokens. It demonstrates that harness choice significantly impacts cost and performance, with the model largely driving cost and 64.5% of spend going to trials that did not pass. The article provides methodology, model-harness pairings, and recommendations to test harnesses with your model and dataset to optimize value.

🚀 Service construit par Johan Denoyer