Game Arena: Strategic LLM Evaluation in Competitive Environments
[Sztuczna inteligencja] Autorzy: Bovard Doerschuk-Tiberi, Yao Yan, Justin Chiu i 59 więcej
We introduce Kaggle Game Arena, an open and ever-expanding platform to evaluate large language models (LLMs) through competitive games. Different from static benchmarks, game arena enables models to play head-to-head matchups in structured environments where the gameplay strength naturally increases as models evolve, preventing performance saturation. This technical report details the infrastructure behind Game Arena and describes the three pilot game environments: Chess, Poker, and Werewolf. These environments span perfect information, imperfect information, and multiplayer game settings, enabling a systematic study of models' strategic planning, adaptation, and robustness under uncertainty. For each game, we provide a detailed description of the environment, evaluation metrics, and results from running full competitions across models. Through robust infrastructure and large-scale grou…
- Kategoria arxiv
- cs.AI
- Kategoria
- Sztuczna inteligencja