Towards Continual Learning: Mitigating Forgetting With Subspace Tuning
[Uczenie maszynowe] Autorzy: Arnav Chavan, Kusha Sareen, Rishabh Tiwari i 6 więcej
Large language models (LLMs) acquire new skills through fine-tuning, but it can destroy capabilities they already have: catastrophic forgetting remains a central obstacle to continual learning in LLMs. Recent works connect forgetting to how far, and along which directions, fine-tuning moves the weights. On-policy methods such as reinforcement learning and on-policy distillation avoid updating the principal singular directions of weight matrices and thus forget less. In contrast, SFT concentrates updates along these directions and is susceptible to catastrophic forgetting. In this work, we introduce Subspace Tuning (SST), which restricts each weight update to the bottom singular subspace, eliminating changes along the principal directions. SST is optimizer-agnostic and composes well with fine-tuning recipes including SFT and self-distillation. SST combines tight subspace protection, dens…
- Kategoria arxiv
- cs.LG
- Kategoria
- Uczenie maszynowe