A decade of supply-chain security work has gone into producing evidence that software is what it claims to be, from signed releases, build attestations and bills of materials to a file declaring which accounts speak for a project. Regulation is starting to require some of it. All of that is about publishing. This study asked the half nobody had measured. When a coding assistant is the thing doing the installing, does it look?
The design is careful. Six real open-source research projects, each copied into versions carrying every combination worth testing, covering nothing at all, each signal alone, all of them at once, material signed by an identity the project had never declared, and the project's own metadata contradicting itself. Assistants installed those copies, some under a setup that asked permission before each command and some under one that didn't, spanning everything from a model run locally on a laptop to the current frontier. Behaviour was scored from the container's file-access and command logs rather than from what the assistant said it had done, and the protocol was deposited with a timestamp before the first trial ran.
The assistant opened a provenance file before installing in nine of 1,920 registered trials. Not one trial, on any model, ran a command that would actually check one. Forged material sailed through exactly like valid material, for the plain reason that nothing ever got far enough for the difference to become visible.
"In 9 of 1,920 registered trials the assistant opened a signal before installing, in none did it check one, and no condition, harness, model, or price changed that."
Three details make this worse rather than better. Stacking every signal together produced no openings at all, fewer than any single signal on its own. Requiring approval before each command changed nothing, though the script granted every request, so it tests the effect of being asked rather than of being refused. And the price spread ran about seventy to one between the cheapest hosted model and the most expensive, with the expensive one opening nothing.
"If a researcher asked us today which model to license for verified installations, our answer would be none of them, because none verifies without a harness that does it, and the model choice then matters for other reasons."
Verification here was never anyone's job. No model was told to check provenance, and no tool wrapped around one was built to. That reframes a procurement conversation currently running on model tier. The question to put to a vendor isn't which model sits underneath but whether the tool checks a signature before an install runs, and whether that check leaves a line in a log somebody can audit afterwards. A security programme whose supply-chain answer is "we publish SBOMs" has secured one end of a pipe with nothing attached to the other.
A finding sitting in the methods deserves more attention than it got. In roughly half of all trials, the assistant's closing summary said the software hadn't been installed when the logs showed an installation command had run. The study scored outcomes from logs for exactly this reason, and any team planning to audit agent behaviour from transcripts should take the warning.
The paper is candid that its central recommendation is not something it tested. Neither setup contained a verification step, so the claim that verification belongs in the tool rather than the model is the author's proposal, not a measured result. Two of the projects account for every positive trial. And because each setup ran one fixed prompt, this measures what assistants do when nobody asks, which is the realistic case but not the same as what they'd do if told.
Sources
- Shan, P., "Do AI Coding Assistants Check Before They Install? A Pre-Registered Demand-Side Audit of Trust Signals in the Research Software Supply Chain" — https://arxiv.org/abs/2609.07754