Generative Tutorial: Towards Live Contextualized Visual Instructions for Physical Tasks
[Interakcja człowiek–komputer] Autorzy: Muzhe Wu, Zuchen Li, Xu Wang i 1 więcej
Visual instructions for physical tasks are typically authored in one context and followed in another, requiring users to translate demonstrated tools, materials, and spatial relationships into their own environment. We introduce Generative Tutorial, a conceptual framework for live visual instruction that depicts intended outcomes and actions within the user's environment and task flow. A formative evaluation of state-of-the-art image and video generation identifies failures and potential benefits across 15 physical tasks. Drawing on these findings, we build an augmented-reality prototype system that proactively generates goal images and demonstration videos using observed workspace context and predicted visual outcomes of preceding actions. A 24-participant lab study found higher task performance quality, greater perceived workspace correspondence, and shorter step-confirmation interval…
- Kategoria arxiv
- cs.HC
- Kategoria
- Interakcja człowiek–komputer