Rollout-Time Intervention: Unlocking Agentic Reinforcement Learning Beyond Policy Limits
[Sztuczna inteligencja] Autorzy: Chen Xi, Ding Ruixue, Wang Zhigaoyuan i 9 więcej
Reinforcement learning (RL) has emerged as a key paradigm for improving large language model (LLM) agents, enabling stronger reasoning, planning, and tool-use in interactive environments. However, existing agent RL methods remain fundamentally constrained by an on-policy reasoning bottleneck: when tasks exceed the model's current capability boundary, effective trajectories become extremely sparse and training often collapses. We introduce Rollout-Time Intervention (RTI), a technique designed to elicit successful trajectories beyond the policy's intrinsic exploration capacity. RTI operates by leveraging reward-side evidence (i.e., evaluation rubrics) during rollout to convert erroneous trajectories into successful ones, providing effective learning signals for tasks that the policy rarely solves on its own. Although intervention trajectories are inherently off-policy, we find that they c…
- Kategoria arxiv
- cs.AI
- Kategoria
- Sztuczna inteligencja