Learning to Stop without Learning to Stop: Self-Supervised Confidence Training Improves Reasoning Efficiency
[Sztuczna inteligencja] Autorzy: Parsa Hosseini, Akasha Tigalappanavara, Sumit Nawathe i 6 więcej
Reasoning models often generate very long reasoning traces, making inference computationally expensive. Existing approaches typically improve efficiency either through inference-time early-stopping mechanisms or by explicitly encouraging shorter reasoning during training, for example through reinforcement learning with length penalties. We show that substantial efficiency gains can instead emerge from a different kind of supervision: \textit{confidence}. Using a self-supervised procedure, we fine-tune reasoning models to predict their confidence in the answer at intermediate points along their own reasoning trajectories using only 600 training problems. Confidence is used only as a training target: the loss contains no objective for reasoning length, efficiency, or stopping. At inference, the fine-tuned models use the standard generation procedure, with no confidence elicitation or earl…
- Kategoria arxiv
- cs.AI
- Kategoria
- Sztuczna inteligencja