RACE: Scalable Statistical Estimation of Functional Consistency in LLM Neurons
[Sztuczna inteligencja] Autorzy: Runyu Wang, Bo Liu, Xiaxin Zhang i 6 więcej
Discovering stable neuron behavior across entire domains remains a challenge in mechanistic interpretability. Existing methods often rely on instance-level point estimates or computationally expensive procedures, which either obscure population-level variability or limit scalable domain-wide analysis. We present RACE (Residual Alignment for Consistency Estimation), a forward-pass statistical framework that evaluates the domain-wide functional consistency of Transformer neurons. Perturbation experiments demonstrate that RACE achieves superior domain specificity compared to gradient-based point estimates. Meanwhile, token-distribution-level results verify the association between the selected neurons and the target domain. Furthermore, its computational overhead is two orders of magnitude lower than that of gradient-based methods.
- Kategoria arxiv
- cs.AI
- Kategoria
- Sztuczna inteligencja