On the Limits of Self-Improving in Large Language Models: The Singularity Is Not Near Without Symbolic Model Synthesis
Summary
Zenil et al. formalize recursive self-training in large language models as a discrete-time dynamical system and show that when external grounding vanishes (αt → 0), the system undergoes entropy decay and distributional drift that converge to degenerate fixed points, undermining unbounded improvement. They demonstrate entropy decay and information-theoretic stagnation under closed-loop, KL-based learning and differentiate it from externally anchored optimization. To counter collapse, they propose neurosymbolic approaches grounded in algorithmic information theory (CTM and BDM) with a three-part update (symbolic projection, causal correction, and statistical fitting) to pursue mechanism-based learning rather than mere correlations.