Puls Dane publiczne, aktualizowane automatycznie
arXiv – nowe publikacje publikacja

Do Verifier Judgments Survive Abstraction? Measuring Verification Transport for LLM Agent Evaluation

[Przetwarzanie języka] Autorzy: Disha Sheshanarayana, Sai Linisha Reddy Goli

LLM-based verifiers are increasingly used to evaluate agent executions, but their reliability is typically measured under a single representation of the trajectory. In deployed agent systems, the same execution may be exposed through composed tools, delegation boundaries, or compacted histories, changing what execution evidence is directly observable to the verifier. It therefore remains unclear whether correct verifier judgments persist across such abstraction boundaries. We introduce Verification Transport, a reliability test that measures judgment stability under abstraction-induced changes in execution observability. We construct matched native, abstracted, and restored views of the same frozen executions while holding the task, realized trajectory, outcome, ground truth, evaluation criterion, and verifier fixed. Across four agent-evaluation benchmarks, seven verifier backbones, fou…

Kategoria arxiv
cs.CL
Kategoria
Przetwarzanie języka
Pdf
https://arxiv.org/pdf/2611.18471v1

Otwórz oryginał w źródle ↗

Więcej z tego źródła

Wszystkie