Beyond Repeated Sampling: Learning Search Policies for LLM Reasoning
[Przetwarzanie języka] Autorzy: Ismail Labiad, Matthieu Kowalski, Marc Schoenauer i 2 więcej
Large language models increasingly tackle hard reasoning problems by spending more test-time compute, yet the dominant strategy remains naive repeated sampling: draw many independent solutions and hope one is correct. Because such sampling explores only through local decoding noise, it tends to produce many near duplicate attempts rather than genuinely different ideas. We ask whether exploration can instead be steered at a semantic level, by first sampling problem specific concepts, hints, or strategies and then conditioning answer generation on them. We refine this into a simple, more exploratory procedure that emits many diverse concepts in a single trajectory, and evaluate it on hard problems where repeated sampling struggles. We then go a step further and make concept generation trainable: a small concept generator is optimized with reinforcement learning so that its concepts maximi…
- Kategoria arxiv
- cs.CL
- Kategoria
- Przetwarzanie języka