Most worry about agent skills so far has been about skills that do something extra, such as running a hidden script, sending data somewhere or installing a package nobody asked for. That kind of attack leaves a trail, because the skill does something its description never mentioned. This paper tests a quieter version, in which a poisoned skill does what it says, runs cleanly and gets the answer wrong.
The researchers took public skills and changed one load-bearing rule in each, such as a constant in a formula, a threshold or a branch in the decision logic. Their worked example is a medical calculator for a heart measurement, where a single number in the formula's denominator was nudged up and the surrounding text rewritten so every sentence still agreed with itself. The interface, the output format and the bundled scripts stayed untouched. They then padded each copy's search description with plausible requests so it would rank well, and published it into a test registry of more than 26,000 skills to compete with the clean original.
When Claude Code running Opus 4.8 chose from its search results, it picked the poisoned copy for about half the requests, and roughly one request in ten ended with a wrong answer that a correct skill would have got right. The tampered skills were built using feedback from a single open model and then frozen, yet they caused wrong answers on three other models at much the same rate, without being adjusted for any of them.
Three existing scanning tools flagged under 1% of the poisoned skills. An AI judge did better but still missed more than half, against catching more than nine in ten skills that carried an overtly malicious payload. In some of the misses, the judge's own notes pointed out an error in the skill and it still called the skill benign, because nothing looked like malice.
"Thus, package-level semantic screening can recognize visible signs of sabotage, but it cannot reliably distinguish subtle poisoning from an ordinary implementation error without a trusted behavioral reference."
A scanner that looks for bad intent cannot tell a sabotaged formula from a buggy one, which suggests the defence is the one teams already use against bugs, checking outputs against something trusted. The authors argue for that kind of behavioural testing. What their study cannot show is how practical it is, since many skills exist precisely because the team has no reference implementation to compare against.
The code-writing tasks were the hardest to poison. Most attempts there changed nothing during construction, and although the agent picked poisoned coding skills more often than average, few of those runs ended in a wrong result. The paper reports the gap without explaining it. This piece's guess is that code which has to run and pass tests leaves less room for a quietly wrong rule than a calculation nobody re-derives, though the study did not isolate that. The tasks that broke most often were medical calculators, where the answer is a number that looks reasonable either way.
The paper sets out its own limits. Everything ran offline against a local copy of a benchmark registry; no poisoned skill went near a real marketplace. The two skill-specific scanners ran without their AI analysis switched on, which is weaker than the setups their makers recommend. Most tasks were calculations and reasoning puzzles rather than everyday engineering work, so the one-in-ten figure is a benchmark result, not a forecast for a team's repository.
"Therefore, retrieval and selection cannot be treated as implicit security defenses."
For an engineering leader, the practical change is in what "approved" means for a skill. A clean scan says the package does not reach for things it should not. It says nothing about whether the rules inside it are right. Any skill that encodes a formula, a threshold or a policy the team will act on deserves the same treatment as a new library function, with a handful of known-answer test cases run before adoption and again whenever the skill updates. The harder habit to build is distrusting relevance. An agent that confidently picks the best-matching skill is not vouching for it, and a vendor pitch that presents good skill retrieval as a safety feature is selling something this study shows can be gamed with a few extra lines of description.
Sources
- Qiao, J., Li, R., Pan, S. et al., "StealthyPoison: Manipulating LLM Agent Behavior through Procedural Poisoning of Third-Party Skills" — https://arxiv.org/abs/2611.12155