LiveGeo: Benchmarking Geographic Agents in Continuously Evolving Worlds
[Sztuczna inteligencja] Autorzy: Pengcheng Zhou, Xuanyu Liu, Yanchen Yin i 6 więcej
We introduce GeoWorld, a controllable and continuously evolving geographic environment platform for LLM agents, and LiveGeo, a benchmark for evaluating reliable decision-making in real-world geographic environments using GeoWorld. Unlike existing evaluations that largely reason and plan over fixed geographic states, GeoWorld is grounded in real-world geographic structure and allows external world states to evolve along their own temporal processes while preserving verifiable state histories. LiveGeo places geographic tasks within this evolving environment, requiring agents to actively acquire task-relevant information, verify whether the evidence supporting prior decisions remains valid, and revise their plans when necessary. Our results reveal that even frontier LLMs struggle with static tasks, with no model consistently excelling across tasks, while strong static performance does not …
- Kategoria arxiv
- cs.AI
- Kategoria
- Sztuczna inteligencja