Accurate but Not Humble: Evaluating Epistemic Humility in LLM Agents under Knowledge Conflict
[Sztuczna inteligencja] Autorzy: Kaiser Sun, Bernal Jimenez Gutierrez, Hongjun Liu i 4 więcej
When retrieved evidence contradicts an agent's prior beliefs, does it revise its answer, acknowledge uncertainty, or persist with an incorrect conclusion? Existing evaluations of agentic systems focus primarily on task success, offering limited insight into how agents handle such conflicts. We propose to evaluate agents on epistemic humility (EH): the agent's willingness to recognize, act on, and communicate uncertainty during task execution. We operationalize EH through three trajectory-level behavioral dimensions: Identify, Solve, and Escalate (ISE). Through knowledge conflict, situations where the backbone language model's parametric knowledge contradicts the evidence it encounters, or where two contextual sources disagree, we evaluate two conflict settings: (1) controlled conflict and (2) naturally occurring conflict during multi-step agentic execution, each paired with matched no-c…
- Kategoria arxiv
- cs.AI
- Kategoria
- Sztuczna inteligencja