Client-Side Load Balancing at a Million Requests Per Second
Summary
Zalando’s engineering blog outlines building an in-process client-side load balancer to handle a million requests per second of internal fan-out traffic. The article details a hash-ring parity approach with Skipper, an N-ring fade-in for scale, occupancy-based bounded-load, and AZ-aware routing experiments, all aimed at reducing latency, cutting costs, and increasing resilience. It also documents pipeline improvements, safety toggles for rollout, and the practical insight that owning routing requires corresponding telemetry and careful production testing.