The intuitive story about coding agents is that they compound. Give one a few months in a repository, let the team learn to prompt it, and acceptance should climb. A study of more than 220,000 closed pull requests from hundreds of popular open-source Python projects, filed by five widely used agents (OpenAI Codex, GitHub Copilot, Claude Code, Cursor, and Devin), tests that story directly, and finds almost no trace of it.
Which agent filed the pull request mattered enormously; how long it had been working in the repository barely mattered at all. Merge rates ranged from 43% for the weakest agent (Devin) to 84% for the strongest (Claude Code), roughly the acceptance rate of human contributors on the same repositories. Some of that gap comes from the tools themselves; some reflects that the weaker performer tends to show up in smaller, less strictly managed projects. Either way, the agent a team picks changes its odds far more than the passage of time does. Split the same pull requests into four stages of a repository's history with an agent, and the merge rate barely moves.
"Overall, although the estimated probability of PRs being merged is the highest in the third quarter, the differences across development quarters are relatively small."
The task-by-task picture complicates the maturity story further. Agents are used overwhelmingly for documentation, dependency updates, and testing (the safe work teams hand over first), and it merges cleanly. But as a repository racks up more experience with agent PRs, acceptance rates for exactly those categories start falling, while riskier work like writing new functions or touching cloud configuration climbs instead. That's close to the opposite of what "the agent is learning the codebase" predicts.
"One plausible explanation is that, as repositories accumulate more experience with agent-generated PRs, developers adopt more conservative review practices for categories that may introduce hidden dependencies, maintenance overhead, or reproducibility concerns."
On raw code quality the news is milder. Agentic and human pull requests looked broadly similar in size, review effort, and time to merge, and agentic pull requests were not more likely than human ones to later be tied to a bug.
None of this supports a pitch that an agent becomes a better teammate over time. The category of work and the choice of agent are doing the explaining, not tenure. That argues against "give it more time" as a rollout plan, and for judging a vendor's "it learns your codebase" claim on evidence, not faith. It also points to a habit teams should break. They tend to loosen review as they gain confidence in a tool, easing off fastest on the categories that felt safest on day one (docs, dependency bumps, routine tests). This data suggests those are exactly the categories where scrutiny should tighten as adoption grows, since familiarity with an agent's normal output is what lets a subtly wrong change slip past a reviewer who has stopped reading closely. The more useful thing for a team to track is merge and defect rates broken out by agent and by task; tenure with a tool explains almost nothing here. This also comes from public open-source review norms, which may not hold inside a private codebase.
Sources
- Mazloomzadeh, I., Morovati, M. M., Khomh, F., "How Do AI Coding Agents Contribute to Software Development? an Empirical Study of Agentic Pull Requests" — https://arxiv.org/abs/2607.21832