Necessary or Sufficient? Evaluating LLM Explanations With Behavioural Evidence
[Sztuczna inteligencja] Autorzy: Urja Pawar, Rajitha Ramanayake, Nabeel Kemal i 4 więcej
LLM decision components that can operate within agent workflows often produce action-relevant recommendations or judgements together with explanations. Operators may use the named factors to monitor a system, diagnose errors, or decide when to escalate an output. Such use assumes that the explanations agree with the component's observable decision behaviour. We test two interpretations of the named factors: necessity, meaning that changing a factor would change the output, and sufficiency, meaning that retaining it while removing other changeable information would preserve the output. We evaluate these interpretations in two synthetic use cases: recommending advisors to clients and judging prompts for harmfulness or risk. Models return an output and the top three factors that most influenced it. Controlled black-box interventions estimate a necessity score for each factor by measuring h…
- Kategoria arxiv
- cs.AI
- Kategoria
- Sztuczna inteligencja