Surprising Economics of Load-Balanced Systems
Summary
Marc Brooker analyzes load-balanced systems with an M/M/c queue behind a load balancer. Using Erlang’s C formula and Monte Carlo simulations, the post shows that increasing the number of servers while keeping per-server load constant yields asymptotic latency improvements and better percentile latency, with only a small fraction of requests queuing even at higher loads. He notes caveats about Poisson arrivals and exponential service time and discusses cloud economics implications.