CLIFT: Conformal Self-Verification for Web Agent Training and Test-Time Scaling
[Przetwarzanie języka] Autorzy: Yifan Zhang, Yutong Dai, Viraj Prabhu i 3 więcej
Open-source web agents are now strong enough to execute realistic browser tasks, but training them with reinforcement learning still depends on weak supervision: binary task success is too sparse for credit assignment, while frontier-language-model judges are too expensive to call at every step and cannot be assumed available at deployment. We introduce CLIFT, a training and test-time scaling method built around conformal self-verification. During training, the agent answers natural-language verification questions about its own rollouts; a Compositional Conformal Certifier keeps only question signals whose URL-conditional evidence agrees with a training-time judge, assigns signed trust weights through polarity-aware lift, and blends the resulting verifier score into per-step rewards in a way that never subtracts from the judge baseline. At test time, the same certified bank is frozen an…
- Kategoria arxiv
- cs.CL
- Kategoria
- Przetwarzanie języka