DigiNews

Summary

The arXiv paper Self-Distillation Enables Continual Learning introduces Self-Distillation Fine-Tuning (SDFT), a method that enables on-policy learning from demonstrations by using a demonstration-conditioned model as its own teacher. SDFT aims to preserve prior capabilities while acquiring new skills, outperforming supervised fine-tuning and reducing forgetting in sequential learning tasks. The work presents a practical path toward continual learning from demonstrations for foundation models.