TurnCost: Turns, Not Observations: Cache-Aware Loop Engineering for Coding Agents
[Inżynieria oprogramowania] Autorzy: Pramod Kumar, Raj Kumar Pandey
AI coding agents operate in iterative loops: they modify code, run tools, inspect the results, and decide what to do next. We call the design of this process loop engineering, which includes context management, model selection, compaction, and termination. Existing efficiency work mainly compresses tool observations or routes selected turns to cheaper models, but the practical opportunity for these optimizations remains unclear. We analyze 600 publicly released coding-agent trajectories spanning two agent scaffolds and two language models. Median observations contain only 118-286 tokens, while the large build logs commonly used to motivate compression occur near the ninety-ninth percentile. Moreover, 98.6% of input tokens are served from a prefix cache, so flat input pricing overestimates loop cost by roughly 7x on average under near-ideal caching. We derive and empirically validate a c…
- Kategoria arxiv
- cs.SE
- Kategoria
- Inżynieria oprogramowania
- Doi
- 10.1145/3843779.3844635