Rules files like CLAUDE.md and AGENTS.md tend to get written once, while the code around them keeps moving. A team switches from print statements to proper logging and nobody goes back to delete the line that says "use print()". A new study from Georgia Tech and the University of Colorado asks what a coding agent does when the file and the code disagree.
The researchers built 40 small Python projects where the rules file asked for one convention and every existing module used the other. Examples include logging style, how money is stored, and how errors get reported. Then they had Claude Code and Codex CLI add a feature to each one. Without help, 90% of the agents' choices followed the stale file rather than the code in front of them.
The two agents got it wrong in different ways. Claude Code named the conflict in its final reply in 79 of 80 runs, and still wrote code that matched the file. Codex CLI rarely mentioned the conflict at all, and in a quarter of its runs it went further, rewriting working modules to match the file. The project's own tests then broke in almost all of those runs.
This is how the researchers describe Claude Code's runs.
"The agent read the code, saw the difference and treated the code as the part that was out of line."
The fix was one sentence in the user's personal rules file. It said that when documentation and code disagree, follow the code and list the inconsistencies. With it, choices that matched the existing code jumped from almost none to 91%, the rewrites stopped, and every test suite passed.
The authors think the cause is how agents receive their inputs. The rules file arrives as an instruction. The code arrives as tool output, which models are trained to trust less so that text planted in a file or web page can't hijack them. The experiment fits that story, but it never tests it head on. Nobody moved the file into tool output to see whether the behaviour flips.
The study can't settle one case, because it was designed to leave it out. Every project made the rules file the stale side. Plenty of teams write a new rule precisely because they want the code to change, as in "we're moving to integer cents, use them from now on." Taken literally, the one-sentence fix tells the agent to side with the old code and ignore the new rule, though the study never ran that case. So the sentence isn't free. It probably trades one failure for another, and which one costs more depends on how often a team uses the rules file to drive change.
These files are doing two jobs at once. Some lines describe how the code is. Others say where it should go. Splitting the two, with migrations stated as migrations, probably lets the agent tell an old description from a fresh order. The study also hands reviewers two cheap checks. Read the agent's final reply, since Claude Code was confessing the problem in plain text that a busy reviewer would scroll past. And treat a feature pull request that deletes large chunks of untouched modules as a sign the agent has been tidying the codebase to match a file nobody updated.
Sources
- Xu, X., Wu, W., "Stale Instruction Files Override the Code in Coding Agents: A One-Sentence Fix" — https://arxiv.org/abs/2610.30571