Do Verifier Judgments Survive Abstraction? Measuring Verification Transport for LLM Agent Evaluation
[Przetwarzanie języka] Autorzy: Disha Sheshanarayana, Sai Linisha Reddy Goli
LLM-based verifiers are increasingly used to evaluate agent executions, but their reliability is typically measured under a single representation of the trajectory. In deployed agent systems, the same execution may be exposed through composed tools, delegation boundaries, or compacted histories, changing what execution evidence is directly observable to the verifier. It therefore remains unclear whether correct verifier judgments persist across such abstraction boundaries. We introduce Verification Transport, a reliability test that measures judgment stability under abstraction-induced changes in execution observability. We construct matched native, abstracted, and restored views of the same frozen executions while holding the task, realized trajectory, outcome, ground truth, evaluation criterion, and verifier fixed. Across four agent-evaluation benchmarks, seven verifier backbones, fou…
- Kategoria arxiv
- cs.CL
- Kategoria
- Przetwarzanie języka