Study finds coding AI agents waste up to 22.75% of task cost on repeats

A new study says three repetitive habits in AI coding agents, software tools that write and fix code on their own, quietly inflate the cost of running them. The paper, from six researchers including Yiran Hu, Nan Jiang and Lin Tan, is posted on arXiv, an open repository for research papers before formal peer review. The team studied 1,200 trajectories, meaning full records of an agent’s steps on a single task, produced by Anthropic’s Claude Code and the open-source Mini-SWE-Agent, tested across four configurations on SWE-bench Verified, a benchmark of real software bugs used to measure how well an agent fixes code.

Three behaviors stood out. Subsumed retrieval happens when an agent fetches information it has already gathered in an earlier, broader search. Similar script generation happens when an agent writes nearly the same throwaway code multiple times instead of reusing it. Test re-execution happens when an agent reruns the same tests repeatedly without changes that would justify it.

According to the paper, these three habits show up in 79.00% to 98.00% of coding tasks and can account for up to 22.75% of a task’s total cost. Because coding agents are billed by computing use, wasted steps translate directly into wasted money for anyone running them at scale.

Fixes tried, mixed results

The researchers then tested three ways to cut the waste, running more than 10,000 trajectories on a separate set of SWE-bench Verified and SWE-bench Pro tasks. Structure-aware retrieval, which gives an agent a map of a codebase’s structure to search more precisely, did not consistently help. The paper says it introduced its own overhead and changed how agents delegated work, in some cases increasing cost by as much as 28.14% instead of lowering it.

Skills, which are written instructions an agent can consult mid-task, worked better, but only when a person wrote them. Skills the agent generated for itself, based on its own past traces, tended to be narrow and tied to the specific task that produced them, which the paper says limited how well they carried over to new problems. Skills written by developers, aimed at general patterns rather than one trace, cut cost by up to 41.73%, roughly twice the best result achieved with agent-written skills.

The paper does not say whether the findings hold for coding agents other than Claude Code and Mini-SWE-Agent, or for tasks outside the SWE-bench benchmarks used in the study. It also does not specify which developer-designed skills produced the largest savings, or whether the authors plan to release the skill files or trajectory data. Teams running coding agents in production can watch for a public code or data release, and for follow-up studies testing developer-designed skills on benchmarks beyond SWE-bench.