Next-Concept Modeling: Fast, Parallel Coarse-to-Fine Text Generation
Summary
The paper proposes Next-Concept Modeling (NCM), a two-stage coarse-to-fine text generation method using a multiscale residual VQ-VAE and a transformer to generate codes level-by-level in parallel. It reports significant speedups over diffusion language models while maintaining or improving quality across benchmarks (LM1B, OpenWebText, XSum) and Sudoku reasoning, suggesting impactful implications for faster NLP inference.