Shockingly Simple Self-retrospection Improves Agentic Models Without RL
[Sztuczna inteligencja] Autorzy: Jonathan Light, Christopher Zhang Cui, Jeonghye Kim i 7 więcej
People learn not only by repeating successful actions, but also by recounting and explaining their experiences, revising their understanding to guide future behavior. Can a language-model agent improve its future actions by training only on explanations of its own experience? We investigate this question by studying Retrospection-Only Fine-Tuning (ROFT), a minimal online procedure designed to isolate the effect of explanation-only training on subsequent behavior. The agent attempts a task, observes available feedback, generates a retrospective explanation, and is fine-tuned with a next-token prediction loss on the explanation tokens alone. The procedure uses neither an external teacher nor a reward-based policy update. In software-engineering experiments with Qwen3.5-4B, ROFT is trained on problems with mixed successful and unsuccessful base-model attempts. On held-out SWE-bench Verifie…
- Kategoria arxiv
- cs.AI
- Kategoria
- Sztuczna inteligencja