Latent Lattice Language Models
Summary
The paper proposes Latent Lattice Language Models that introduce per-layer horizontal latent streams, or latents, which persist across tokens to augment the vertical residual stream. This persistent state shows perplexity improvements on large-scale pretraining settings and integrates with existing residual enhancements, offering a new axis for scaling LLM computation.