EpiSEM-Bench: Evaluating Safety and Evidence Recovery in LLM Agents under Missing Evidence
[Sztuczna inteligencja] Autorzy: Qijin Zhang, Hanzhen Wang, Yuehan Zhao i 4 więcej
Tool-using language model agents rely on environmental facts to decide whether an action is permitted. When a tool response omits a safety-relevant fact, an agent may carry out a prohibited action even though the user request is benign and every returned statement is true. We introduce \bench{}, an executable benchmark for evaluating unsafe execution, safety boundaries, and evidence recovery in LLM agents under missing evidence. It contains 300 matched unsafe/safe task pairs across six domains. By systematically suppressing critical facts, the benchmark measures unsafe execution under critical evidence omission and determines the minimum observed failure cardinality, including failures with full evidence and cases with no observed failure after exhaustive testing. An independent audit tool tests whether agents seek missing facts and then withhold prohibited actions while completing perm…
- Kategoria arxiv
- cs.AI
- Kategoria
- Sztuczna inteligencja