From Parameters to Answers: How LLMs Retrieve and Use Their Internal Knowledge
[Sztuczna inteligencja] Autorzy: Wenkang Wei, Yuan Fang, Renhe Jiang i 2 więcej
How does a language model's dependence on query-routing information and target knowledge change as it answers a question? We study this question through layerwise interventions on the hidden state at the end of the question. Across Qwen, Llama, and Gemma, we compare country-continent questions with noun, adjective, and code answers while keeping several fitted measurements distinct. A pair-conditioned request direction describes which country is queried in natural single-country questions; a global request direction describes first- versus second-country requests in paired questions; separate selection candidates test control among contents already available in the hidden state. A diagnostic reanalysis of frozen Qwen natural-question states shows that the pair-conditioned direction grows stronger before interventions on it begin to alter later fitted knowledge, with this causal window o…
- Kategoria arxiv
- cs.AI
- Kategoria
- Sztuczna inteligencja