Cross-request KV cache exposure in the GLM-5.3 Workers AI model, and our remediation with SGLang
Summary
Cloudflare reports a cross‑request KV cache exposure in the GLM-5.3 Workers AI model affecting 155 responses. The incident was caused by a misconfiguration of the SGLang inference engine and a bug in handling prompts longer than the KV pool, with a pool size of 681,280 tokens vs the 1,048,576 token context. Cloudflare remediated by aligning pool size to 2.04 million tokens, patching the engine to reject over-length requests earlier, and applying upstream fixes with SGLang; coordinated disclosure followed with recommendations for operators deploying disaggregated AI inference workloads.