Regression tests exist to freeze behaviour. If a change breaks something that worked, a test fails and somebody looks. That bargain depends on the test having captured what the code was meant to do, and an AI test generator can't see that. It reads the implementation in front of it, infers the intent from whatever the code happens to do, and then, by the design rule nearly all these tools share, keeps only the tests that pass on the code it just read.
A team at the University of Luxembourg worked out what that costs. They took merged pull requests from SciPy, Qiskit and pandas, planted a small bug in the code each one changed, kept only the bugs the projects' own test suites missed, and had a standard change-aware generator write regression tests for the result. Between 8% and 17% of the tests it produced passed only because the bug was present, and failed once the code was corrected. The share that caught the bug was several times smaller. In 63% of the pull requests, at least one planted bug ended up written into the suite as expected behaviour.
"When a faulty PR is accepted and merged, a generated test can become fault-enforcing: it passes silently on the faulty code, giving false confidence, and it fails once the fault is fixed, making the fix harder."
Then they followed those tests forward through each project's later history. More than eight in ten were still endorsing their bug at the end of the record, and almost none ever came to detect it. In two of the three projects, the developers' own tests rarely contradicted them. Piling up to two dozen latent bugs into one codebase didn't shake the wrong expectations loose either.
"We observe the opposite, a test that expects the faulty behavior keeps expecting it when the surrounding code carries a dozen other faults."
The authors put this down to two causes, a generator reading intent off the implementation, and the rule that a test is kept only if it passes. Their design can't separate the two, and they say so plainly. That matters, because the second is a product decision somebody could change next quarter and the first is a hard problem. They're also careful about their own limits. These were mechanical, injected bugs, and treating an enforced one as a real defect locked in is an interpretation.
A green suite certifies nothing about intent once its tests are recordings of current behaviour. Thousands of them lift coverage and test count while answering a different question from the one a reviewer thinks they answer, and the failure is silent on the day it's created, loud months later in the hands of whoever fixes the bug. On this evidence, the review habit that matters is reading a generated test's assertions against the ticket, never against what the code currently returns. It also hands you a sharp question for any vendor quoting coverage lift. What share of these tests can tell a correct implementation from a broken one? Here, most couldn't tell them apart at all.
Where this will get over-read is as an argument against letting AI write tests. The generator did write tests that caught the planted bug. Current tooling throws those away, because a failing test gets treated as a test to repair rather than a bug to report. That's the cheaper fix, and nobody needs a new model for it.
Sources
- Charoosaei, M., Richter, C., Papadakis, M., "On the Risks of using LLM-Generated Tests for Regression Testing" — https://arxiv.org/abs/2610.11835