AgentSWE: Can Coding Agents Build the Agent You Actually Want?
[Inżynieria oprogramowania] Autorzy: Jiahao Wang, Hongjin Qian, Yuyang Hu i 2 więcej
As the production of software passes into the hands of software itself, coding agents are increasingly commissioned to build agents fitted to their users' own workflows. This delegation carries a quiet assumption, that the delivered agent will faithfully serve the requirement it was built for, rather than merely appear to. No existing benchmark tests that assumption. We introduce AgentSWE, a benchmark for agent software engineering spanning three stages of the agent development lifecycle: Creation of a complete agent from a natural-language requirement, Editing of production agent codebases, and Optimization toward a measurable target. Each of its 25 tasks is framed as a commission rather than an exam: the builder iterates against a handful of illustrative cases, while acceptance exercises the frozen submission on held-out cases designed to span the full requirement, grading behavior ag…
- Kategoria arxiv
- cs.SE
- Kategoria
- Inżynieria oprogramowania