LAPT-Scale: Scaling Embodied Policies with Latent Action Pre-Training on Human Videos
[Widzenie komputerowe] Autorzy: Zhiyuan Xiang, Peidong Liu, Mingyang Li i 4 więcej
Scaling generalist robotic policies is limited by the high cost of collecting embodied interaction data. Large-scale egocentric human videos offer a rich alternative, but they lack robot action labels, and existing action annotation methods are often noisy under real-world visual complexity and diverse manipulation dynamics. We introduce LAPT-Scale, a scalable latent action pre-training paradigm with two components: LAPT-Scale-LAM learns continuous latent actions by explaining frame-to-frame visual transitions in human videos and uses them as human-derived supervision, while LAPT-Scale-Policy trains a VLA policy through human-video latent-action pre-training, target-robot embodiment alignment, and task-specific post-training. Unlike prior vector-quantized LAMs, LAPT-Scale-LAM uses a continuous latent space to preserve fine-grained motion and interaction dynamics. We further combine perc…
- Kategoria arxiv
- cs.CV
- Kategoria
- Widzenie komputerowe