WorkflowArena: Benchmarking Executable Workflow Orchestration in LLM-Based Multi-Agent Systems
[Przetwarzanie języka] Autorzy: Zheng Jiang, Fang Niu, Bo Peng i 10 więcej
Large language model agents are increasingly deployed as multi-agent systems rather than as a standalone model operating within a single context. Recent agent harnesses allow the model to instantiate worker agents and to author the executable workflow that coordinates them, shifting the design of system structure from developers to the model at run time. The quality of this organization therefore constitutes a distinct capability, yet existing benchmarks either measure end-to-end task completion or evaluate coordination under structures specified in advance. We formalize this capability as workflow orchestration and introduce WorkflowArena, a benchmark of 145 long-horizon tasks spanning 8 professional domains, from software engineering and formal mathematics to legal analysis and 3D production. Each task specifies an initial workspace and an integrated deliverable composed of interdepen…
- Kategoria arxiv
- cs.CL
- Kategoria
- Przetwarzanie języka