Puls Dane publiczne, aktualizowane automatycznie
arXiv – nowe publikacje publikacja

One Prior, Two Dynamics: Repurposing Video Diffusion Transformers for Self-Supervised Latent Action Learning

[Widzenie komputerowe] Autorzy: Yuxin Jiang, Wenqiang Sun, Jiaqi Li i 4 więcej

Videos reveal how the world changes, but rarely the actions that cause those changes. Latent action learning recovers such supervision from visual transitions, yet often learns the underlying dynamics largely from scratch. We ask whether a pretrained video generator can instead provide the prior for both inverse and forward dynamics. We introduce O$^{2}$LaF, built on a simple principle: one prior, two dynamics. We initialize both models from the same pretrained video DiT and adapt them to their distinct roles: a feed-forward transition encoder with learnable action queries for inverse dynamics, and a causal, action-conditioned video generator for forward dynamics. By controlling information access and optimizing both dynamics end-to-end through a compact bottleneck, O$^{2}$LaF learns transferable actions from unlabeled videos. Across navigation and robotic manipulation, O$^{2}$LaF sets …

Kategoria arxiv
cs.CV
Kategoria
Widzenie komputerowe
Pdf
https://arxiv.org/pdf/2611.26617v1

Otwórz oryginał w źródle ↗

Więcej z tego źródła

Wszystkie