SciExam for ENSO: Can AI Agents Build Climate Models?
[Sztuczna inteligencja] Autorzy: Yinling Zhang, Langchen Liu, Dongbin Xiu i 4 więcej
Language-model agents are increasingly asked to carry out open-ended scientific research, yet their results are usually graded against a known answer, a rubric, or a language-model reviewer, none of which can tell whether a new scientific model is valid. The AI Science Exam for El Nino-Southern Oscillation (SciExam for ENSO) is a benchmark in which agents build low-order stochastic models of ENSO, the dominant mode of interannual climate variability, from real observations. Within a six-hour budget, agents process the observations, write their own diagnostics, which are then frozen, and develop a model using only these diagnostics as feedback. Hidden graders then test whether the model reproduces ENSO's statistics, recovers unobserved variables, and forecasts held-out years, and score a published model in the same way. Across twelve agent systems, six produce models that score higher th…
- Kategoria arxiv
- cs.AI
- Kategoria
- Sztuczna inteligencja