Puls Dane publiczne, aktualizowane automatycznie
arXiv – nowe publikacje publikacja

AURA: AUdio-dRiven streaming Avatar

[Widzenie komputerowe] Autorzy: Nathaniel Cohen, Nicolas Dufour, Amélie Royer i 2 więcej

Current audio-driven video diffusion models can synthesize realistic talking-head videos but generally require dozens of diffusion denoising steps over multi-second video chunks, making them unsuitable for streaming applications such as voice assistants and virtual avatars, where low-latency and online causal generation are required properties. AURA generates video chunk by chunk with a rolling KV cache, requiring only the audio of the current chunk to start generating, rather than a multi-second audio look-ahead like bidirectional models. It comes in two sizes: AURA 1.3B streams 576p video in real time on a single GPU (20.7 fps for a 16 Hz playback rate), making it suited to interactive avatar applications, and AURA 5B streams 720p video at 9.7 fps. Our two-stage distillation pipeline first aligns the causal student with the teacher, by training it to match the teacher's trajectories w…

Kategoria arxiv
cs.CV
Kategoria
Widzenie komputerowe
Pdf
https://arxiv.org/pdf/2611.26540v1

Otwórz oryginał w źródle ↗

Więcej z tego źródła

Wszystkie