An Empirical Study of Harness Design for Coding Agents
[Sztuczna inteligencja] Autorzy: Run-Ze Fan, Zihao Zhang, Simin Ma i 6 więcej
Coding harnesses shape how autonomous coding agents translate model capabilities into long-horizon software-engineering performance, yet existing work typically evaluates harnesses as monolithic systems, leaving the effectiveness of individual components unclear. To enable component-level comparisons, we study this question with a lightweight coding harness whose execution loop is fixed while three components are varied: planning, action space, and context management. Across four models evaluated on SWE-Bench Verified and Terminal-Bench 2.1, we evaluate 176 matched settings spanning five context-management strategies, four context-window budgets, and targeted ablations of planning and action space. We find that: (1) Context management becomes increasingly valuable as the context-window budget tightens, with most of its benefit coming from preventing context-overflow failures. (2) Stagin…
- Kategoria arxiv
- cs.AI
- Kategoria
- Sztuczna inteligencja