Rethinking Circuit Evaluation: Do Circuits Explain Model Errors?
[Uczenie maszynowe] Autorzy: Li Zhang, Chuqin Geng, Mark Zhang i 4 więcej
Mechanistic interpretability (MI) aims to explain a model's behaviour through analyzing its internal computations; circuit-based explanations aim to isolate these computations with compact subnetworks validated by ablating the rest of the model. We show that circuits validated this way may fail to recover the underlying mechanism of the model's behaviour by closely reproducing its successful decisions while failing to account for most of its errors. Such explanations should account for the model's particular errors as well as its successes. We evaluate this requirement by measuring exact answer agreement separately on model successes and failures, across circuit sizes and ablation settings, on IOI, Docstring, and six model-task settings from the Mechanistic Interpretability Benchmark. We discover that many tested circuits closely replicate correct behaviour while missing most of the mod…
- Kategoria arxiv
- cs.LG
- Kategoria
- Uczenie maszynowe