FinAutoRubric: Expert-Guided Automatic Rubric Generation for Evaluating Financial Research Agents
[Sztuczna inteligencja] Autorzy: Hoyoung Lee, Suyeol Yun, Jack Haverty i 17 więcej
Evaluating finance research agents requires rubrics that reflect expert standards and fix the values correct as of an information cutoff. Expert-reviewed finance benchmarks rely on fixed, per-item rubrics, which are costly to extend and cannot encode each institution's own standard. In FinAutoRubric, experts specify reusable evaluation guidance, while agents and code carry out query-specific rubric generation, review, and validation. This expert guidance governs every agent, as prompts and as rules that code enforces, and a Task Bank of reusable criteria carries it across tasks. In long-horizon loops that follow the expert guidance, a writer agent researches every expected value and a reviewer agent verifies it, and failures escalate to a human. On three expert-authored finance benchmarks, its rubrics track expert scoring as closely as the strongest evaluated generator while stating the…
- Kategoria arxiv
- cs.AI
- Kategoria
- Sztuczna inteligencja