Every coding agent that reads an AGENTS.md or CLAUDE.md file reads it at the start of every session. That makes each line in it a standing instruction, and like any documentation it drifts. A folder moves, a command changes, and the file keeps describing the old layout.
North Carolina State University researchers took three real outdated lines from active open-source projects: a test command pointing at a folder that had moved, a pointer to a file that no longer held the code it named, and a step-by-step procedure that put new code in a directory that had since been split out. They ran Claude Code with two of Anthropic's models on a set of bug-fixing tasks, each under three versions of the instruction file: the line left stale, the line corrected, and the line deleted. Nothing else changed between runs.
Agents followed the stale line when the task led them through it. With the old test command in place, they ran it in 78 of 80 runs. They followed the outdated procedure in about two thirds of runs. The outdated file pointer they never used, because none of those tasks needed it. And they took the stale path silently, almost never mentioning that the instruction file was where it came from.
Because the test folder no longer existed, the documented command collected no tests. But the agents habitually piped test output through a tool that shows only the last few lines, so the shell reported success. In 19 of 40 runs with the stronger model, the first check of the fix ran no tests at all, and nothing in the agent's tooling flagged it as an error.
"Agents fixed the bug in nearly every run, whatever the condition, but a stale line changed how they got there: they ran a test command that tested nothing, which was even reported as a success, or searched a folder that no longer exists."
The agents did recover, usually within a couple of steps, mostly by searching the repository. The cost the researchers could measure was modest: for the stronger model on the test command, about two extra steps (commands, searches or file reads) per run, a third more than with the corrected line. For the other cases and the other model, they could not detect a difference in effort. And not one run, in any condition, edited the instruction file or recommended fixing it. The task asked for a bug fix, not documentation upkeep, which the authors note, so this shows what agents do by default rather than what they would do if asked.
"An agent's workaround leaves a record only in a transcript that maintainers rarely read."
The finding most teams can act on today is the comparison between fixing a line and deleting it. Deleting the stale line removed most of the wrong turns, but only correcting it removed all of them. Without the line, agents in the procedure case still guessed the old folder from the shape of the code, and the stronger model spent as many steps working out how to run the tests as it did following the broken command.
"Deleting a stale line removed most of its effect, but only correcting it removed all of it."
What the study could not establish is the bill. It measured what one encounter with a stale line costs, not how often everyday work hits that line, and that frequency decides whether the cost is noise or a real share of a team's agent spend. A line on a step nearly every task shares, like the command that checks every fix, is the obvious candidate for the largest bill, but that is the authors' expectation, not something the experiment measured. Three cases from one vendor's models in one harness cannot settle how common or how costly this is across a codebase.
For an engineering leader, the silent test run is the result to worry about. On this evidence, a stale instruction file is one easy way to get a verification step that verifies nothing and reports success, which makes it a review problem more than an efficiency one. A team that judges its agents by whether the pull request goes green will see nothing here, since every condition passed. Instruction files belong in the same maintenance loop as build scripts: when a refactor moves a directory, the file that tells agents where things live should be in the same diff, and a simple check that every path it names still exists is cheap to add to CI. It is also reasonable to ask agent vendors why a test command that collected nothing can be reported as a pass, and to treat any claim about an agent's self-verification with that case in mind.
Sources
- Mehtiyev, T., Tadesse, B., Arkoh, L., Assunção, W., "When Instruction Files Go Stale: Documentation Debt and Hidden Cost in Coding Agents" — https://arxiv.org/abs/2611.09802