Overcoming Prior Barriers: Supervised Fine-Tuning under Long-Tail Distribution
[Sztuczna inteligencja] Autorzy: Haohui Wang, Jiahao Xu, Wangzhi Zhan i 9 więcej
Supervised fine-tuning (SFT) adapts pretrained large language models (LLMs) to downstream tasks, but the required concepts can receive substantially different levels of pretrained support. Frequent concepts are more likely to be well learned, whereas rare concepts may remain weakly represented. We introduce a novel notion named prior barrier to quantify how strongly the pretrained model supports competing concepts over the target concept. We observe that prior barriers follow a long-tail distribution, placing head and tail concepts at different starting points for SFT: head concepts face lower prior barriers, whereas tail concepts require additional instructions to overcome their higher prior barriers. Our theoretical analysis further derives a predictive risk bound for SFT under long-tail prior barriers, explicitly characterizing how the prior barrier and accumulated SFT evidence joint…
- Kategoria arxiv
- cs.AI
- Kategoria
- Sztuczna inteligencja