WorldAuditBench: Interactive 3D World Auditing with Multimodal Agents
[Sztuczna inteligencja] Autorzy: Ziyan Jiang, Jingbo Yang, Jiabao Ji i 5 więcej
As interactive 3D worlds are increasingly used to study intelligent behavior, it becomes important to develop efficient pipelines for identifying anomalies in these simulated environments, such as floating objects, traversable walls, or objects inconsistent with the surrounding scene. Multimodal AI systems, including vision-language models (VLMs) and vision-language-action models (VLAs), have shown potential for automating this task. However, 3D world auditing is complex, requiring the close coupling of two distinct capabilities: action, to navigate the 3D world and search for anomalies systematically and efficiently; and visual reasoning, to understand the environment and identify anomalies from multimodal observations. It remains largely unexplored whether multimodal agents can effectively couple these two capabilities, using visual reasoning to identify potential anomalies while taki…
- Kategoria arxiv
- cs.AI
- Kategoria
- Sztuczna inteligencja