Exploring Speculative Decoding in vLLM on AMD GPUs
Summary
The blog post explains speculative decoding in vLLM and compares five drafting methods (Native MTP, Gemma 4 MTP, EAGLE-3, DFlash, DSpark) for accelerating LLM serving on AMD ROCm GPUs. It presents throughput measurements across models and workloads, discusses memory considerations, and provides practical guidance for enabling and tuning speculative decoding in production. The work demonstrates substantial throughput gains on certain configurations while highlighting trade-offs and the importance of workload-driven benchmarking.