MTG Bench: Testing how well LLMs can play Magic
Summary
MTG Bench evaluates how well various LLMs can play Magic: The Gathering using a simulated environment with an MCP server and library tool calls. The post ranks models by performance and cost, discusses architectural decisions like token caching and agent loops, and highlights both successes and common failure modes. It also mentions future directions and provides links to the underlying project and benchmarks.