The efficient frontier of LLM inference
Summary
The article explains the 'efficient frontier' concept for LLM inference, distinguishing techniques that trade latency, throughput, or quality from those that push the frontier outward. It covers batching, parallelism, quantization, kernel optimization, speculative decoding, and disaggregation as practical strategies, with references to Baseten's inference tooling.