Less Decoder is More Encoder: Geometric Representation Learning from Novel View Synthesis
[Widzenie komputerowe] Autorzy: Keerthi Kaashyap, Dennis Anthony, Akshay Krishnan i 5 więcej
This paper examines the role of Novel View Synthesis (NVS) in geometric representation learning. In principle, NVS should reason about 3D scene structure, thereby enabling transferable multi-view geometric representations. Yet, existing encoder-based NVS methods yield poor representations. This is not because of a lack of supervisory signal, but rather due to inconspicuous architectural choices: \textit{spatially expressive decoders} that dilute representational capabilities of the scene encoder, and \textit{low-level pixel-space targets} that hinder feature learning. We present SNAP, a self-supervised encoder-decoder transformer that addresses both through a pose-conditioned local decoder and a latent-space reconstruction objective. SNAP is task agnostic, and we show that it is competitive with special-purpose geometry-supervised methods. SNAP also performs competitively against self-s…
- Kategoria arxiv
- cs.CV
- Kategoria
- Widzenie komputerowe