Probing Spatial State Representations in Visual Foundation Models
[Widzenie komputerowe] Autorzy: Baiqiao Yin, Zhongang Cai, Ruisi Wang i 3 więcej
An observer sees only part of the surrounding world, yet must keep track of what lies beyond the current view. Spatial perception therefore involves more than recognizing individual images: it calls for a persistent spatial state that is updated under self-motion and used to guide action. Do visual foundation models provide representations that support such a state? We study this question through four levels of spatial capability---spatial structure, object persistence, predictive spatial update, and actionability---using frozen-feature probes on ten representations under streaming egocentric observations of static indoor environments. The resulting capability profiles vary across tasks, network layers, and prefix lengths: strong readout of the visible scene does not ensure accurate object persistence or future-feature retrieval. Matched controls separate readout accuracy from reliance …
- Kategoria arxiv
- cs.CV
- Kategoria
- Widzenie komputerowe