Beyond the Verdict: Agentic Code Judging with Verifiable Feedback
[Sztuczna inteligencja] Autorzy: Zheye Deng, Sha Li, Changlong Yu i 4 więcej
Test-time scaling for large language models (LLMs) can involve generating multiple candidate solutions for selection or revising an earlier attempt. To guide selection and revision, we need a verifier that can estimate candidate correctness and provide evidence of failure. In competitive programming, however, official judging systems provide authoritative verdicts under finite submission budgets, without revealing failure explanations or failing inputs. To provide informative feedback without consuming official submissions, we formulate a task in which a model judges and diagnoses a candidate program using only the problem statement and source code, with generated counterexamples serving as checkable evidence. To evaluate this task, we build USACO-JUDGE from the USACO 2026 contests, using accepted and unaccepted programs written by human experts and multiple LLMs. We then train ARBITER,…
- Kategoria arxiv
- cs.AI
- Kategoria
- Sztuczna inteligencja