Google updates Android Bench with new LLMs, but Gemini still lags behind
Summary
Google updates Android Bench with eight new LLMs, including Claude Fable 5, Claude Sonnet 5, Claude Opus 4.8, GLM 5.2, Kimi K2.7 Code, MiniMax M3, Qwen 3.7 Plus, and Qwen 3.7 Max, while Gemini 3.1 Pro remains behind GPT-5.4 and Claude models. The update introduces a Harbor-based testing framework and highlights cost considerations, with notable differences in per-run costs across models.