DigiNews

Summary

DiffusionGemma introduces an experimental open-weight language model that uses discrete diffusion to generate text in parallel blocks, achieving much higher inference speed than autoregressive models. The model fine-tunes Gemma 4 with 3.8B parameters activated and 25.2B total, via a two-stage training pipeline, delivering around 1,500 tokens per second on an H100 while retaining capabilities like long contexts and multimodal inputs. This work explores diffusion-based decoding and hints at hybrid diffusion-AR approaches for future fast LLMs.