Two researchers at Monash University went through tens of thousands of pull requests opened by five coding agents (Codex, Copilot, Devin, Cursor and Claude Code) across open-source projects, and asked a simple question of each one. When the agent changed code, did it touch a single documentation file in the same pull request? 63% of the time, it did not. Every agent did this in roughly half to three-quarters of its pull requests. Refactors were the worst case, with about three in four skipping the docs, against just over half for new features.
The paper is careful about what that number means.
"Lacking a human baseline, we do not claim this rate is specific to agents."
On bugs, the obvious way of counting gave the wrong answer. Counted per pull request, documented changes looked worse, followed by more bug-fix PRs touching the same files. But documented pull requests were also nearly four times larger, and a bigger change simply gives later fixes more files to land on. Measured one file at a time, the picture flipped, and an undocumented file was slightly more likely to need a follow-up fix than a documented one, over every window tested.
"Drift may thus be an indicator of defect-prone change: the file that skipped documentation tends to be the one more likely to come back."
The authors offer this as an interpretation and say the data can't settle why. Writing the docs might help the agent think the change through, or skipped docs might just mark rushed edits, and task type or the repository could explain both. The study also can't show that a later fix had anything to do with the earlier change, and the gap it found is only a point or two.
The neglect also compounds, and that result is firmer. A file an agent changed without documentation was documented again within three months 24% of the time, against 48% for files that got documentation the first time, and the gap survived comparisons inside the same repository.
For an engineering leader, the per-file reversal matters more than the drift rate. Any dashboard or vendor slide counting "fixes after AI pull requests" per PR is mostly measuring how big the PRs were, and on this evidence the same data can tell opposite stories depending on how it's sliced. Ask which way a defect claim was counted before believing either direction.
The authors' own advice is to trigger human review on how big a change is, not on whether it came with docs. The drift finding adds that documentation is something a team has to ask for, because the agent won't reliably volunteer it. A pull request template or review checklist that asks "does this change documented behaviour?" costs nothing and targets refactors and fixes, the work agents document least. The likely over-read is that an undocumented agent change is a buggy one. Nothing in this study shows that skipping the docs causes the later fix, and its measure is blunt besides. A stray CHANGELOG line counted as documenting, while docstrings, inline comments and wikis weren't seen at all. Nobody here checked whether the docs that did get written actually described the code.
Sources
- De Michele, W., Tantithamthavorn, K., "Documentation Drift in AI-Generated Pull Requests" — https://arxiv.org/abs/2610.27381