The linter backlog is the least controversial work in software. It flags the small stuff, like a variable nobody reads, a parameter copied when it could have been borrowed, or a convention broken. That backlog is exactly the sort of chore a team is glad to hand to a model. Nobody argues over a tidy-up commit.
A team at Peking University built a benchmark of files that already worked, each sitting on an unusually complete test suite, ran three industry-standard linters over them, and gave a model the file, its list of warnings, and one instruction, to clear the warnings without changing what the code does. The strongest models cleared roughly nine warnings in ten and left about a third of the files failing tests they had passed before.
What the breakage looked like matters more than the rate. Most failures were not crashes. The file compiled, ran, and returned the wrong answer. In one example from the paper, asked to fix a warning about integer division, a model changed the type of the variable holding the result and left the division itself untouched. The warning disappeared. The defect didn't.
"A revision can be syntactically plausible and even pass a limited test while still failing to implement the intended semantic change."
The researchers then ran the same models on the same files with the same tidy-up instruction, but withheld the list of warnings. Not shown the warnings, the strongest model left 95% of files working, while clearing about half as many of them. The list was what did the damage.
"Explicit lint alerts therefore serve as localization signals and intervention signals: they tell models where to edit and what to optimize, but they also increase pressure to modify code that already satisfies functionality preservation."
That is the authors' reading, and their design supports the first half of it more firmly than the second, because it varies whether the list is there, not what the model does with it. What the edits looked like is suggestive, though. On average, a cleanup added and deleted about as many lines as the file had in it to begin with. Asked to delete one unused variable from a fifteen-line function, a model returned a forty-two line rewrite, warning gone and three new ones in its place.
For anyone deciding where review effort goes, the implication runs against instinct. The cleanup commit needs more scrutiny than the feature commit, not less. It's the change where a model rewrites the most code for the least stated purpose, and the one a reviewer is most likely to skim because the diff looks cosmetic.
It also breaks the obvious way of checking the work. For the strongest models, files that failed scored about as clean on the lint report as the ones that still worked, so on this evidence a falling warning count says nothing about whether the code survived. Any dashboard or vendor claim built on code-health scores, warning counts or alerts-resolved is quoting a number that carries no information about whether the code still works. The gate has to be the test suite, and a cleanup run without one is unreviewable in practice.
Nor does iterating out of it work. Re-running the linter and feeding the remaining warnings back for a second and third pass cleared more of them and left slightly fewer files working by the third round.
"Thus, repeated linter feedback helps remove alerts without resolving the functionality trade-off."
The honest limit is the benchmark itself. These are self-contained competitive-programming solutions with no callers elsewhere, so the study cannot say what the same cleanup does to a file other code depends on. The authors are clear that passing the tests is not the same as behaving identically, either. But the likeliest over-reading is to treat a third as a production breakage rate, and that error points the wrong way. These regressions were caught at all only because the test coverage was close to exhaustive. In a repository with ordinary coverage, the same edits would not break fewer files. They would break them quietly.
Sources
- Deng, T., Zhu, J., Cao, Y. et al., "LintEval: Benchmarking LLMs on Linter-Guided File Refactoring" — https://arxiv.org/abs/2610.20027