Puls Dane publiczne, aktualizowane automatycznie
arXiv – nowe publikacje publikacja

Can Mechanistic Interpretability Help Explain LLM Agent Failures? A Case Study on Web Element Hallucination

[Sztuczna inteligencja] Autorzy: Mohamed Aghzal, Mor Geva, Gregory J. Stein i 1 więcej

Mechanistic Interpretability (MI) has made significant progress in explaining the internal computations of Large Language Models (LLMs); however, these advances have yet to demonstrate comparable success in explaining the decision-making processes of LLM-based agents. This paper examines the challenges MI tools face when applied to realistic agentic failures. We focus on one such failure: web agents hallucinating actions that are not available on a website. We apply a series of mechanistic analyzes to Qwen2.5-7B-Instruct as it interacts with live websites. We find that a linear probe trained on the last prompt token can predict hallucinations before generation begins (0.80 AUC) and steering along a probe direction corrects 85\% of hallucinations, but disrupts 7\% of valid actions. However, these results only provide a partial explanation of why the agent hallucinates. The probe directio…

Kategoria arxiv
cs.AI
Kategoria
Sztuczna inteligencja
Pdf
https://arxiv.org/pdf/2611.18514v1

Otwórz oryginał w źródle ↗

Więcej z tego źródła

Wszystkie