Argo-Bench: Evaluating Data Agents on Enterprise-Scale Workflows
[Przetwarzanie języka] Autorzy: Gabriel Tomitsuka, Arman Raayatsanati, Emma Xing i 2 więcej
Real-world enterprise data science and analytics workflows require reasoning across dozens of tables, performing statistical analyses, and acting on the results. Established text-to-SQL benchmarks evaluate query generation alone, and audits have found their answer keys frequently wrong. Because real enterprise warehouses are too sensitive to release, these benchmarks are built on public datasets where a business event fits in a single table. We introduce Argo-Bench, an evaluation framework comprising 210 data science and analytics tasks. Drawing on public data, peer-reviewed industry literature, and regulatory filings, we simulate a food delivery platform in New York City at true scale, with 81 million orders in 2024, grounded economics, fraud patterns, and marketplace incentives. We export this world to an ERP warehouse of 235 tables and 7.5 billion rows, modeled on the Oracle E-Busine…
- Kategoria arxiv
- cs.CL
- Kategoria
- Przetwarzanie języka