Explorative Modeling -- Unlocking a Third Pretraining Axis and End-to-End Generation
Summary
Explorative Modeling (XM) introduces a third pretraining axis by training with exploration over multiple candidate generations, enabling end-to-end generation and improving data, compute, and parameter efficiency. The approach, termed Mode Forcing, shows consistent gains across image, video, and language domains and suggests end-to-end generation becomes more viable as generative expressivity is scaled.