Watch out for cache read costs
Summary
The article explains that cache reads are often the largest ongoing cost in agentic LLM workloads, particularly as context windows expand. It analyzes how cache architecture (KV caches, RAM vs NVMe) and pricing cliffs impact total cost, and suggests that reducing the number of tool calls can significantly cut expenses. It also highlights market dynamics and the potential for cache-read pricing to become a major profit center for providers.