NeuronEye: Query-Guided Visual Concept Activation for Vision-Language Reasoning
[Sztuczna inteligencja] Autorzy: Ruiyu Yan, Bowen Chen, Shaowen Wan i 1 więcej
Current vision-language models (VLMs) encode visual information in dense hidden states where object identity, spatial layout, and local attributes are implicitly entangled rather than explicitly disentangled, limiting their ability to isolate and modulate the specific visual evidence required by a given language query. Inspired by sparse population coding and top-down modulation in biological vision, we introduce NeuronEye, a plug-in framework that constructs a sparse, concept-level neuron vocabulary from intermediate VLM representations and selectively activates query-relevant visual concepts during inference. NeuronEye decomposes vision-token states into an overcomplete sparse basis organized by concept-level clusters, uses the language query to activate relevant clusters and localize the patches where selected concepts are expressed, and injects the focused evidence back into vision …
- Kategoria arxiv
- cs.AI
- Kategoria
- Sztuczna inteligencja