An agent skill is a folder with a markdown file in it, telling a coding agent how to do some recurring job. Teams now write them the way they used to write runbooks. Researchers collected 138,000 of these files from public repositories and ran static checks over each one against the format's own published rules.
Nine in ten had something wrong. The authors are careful to say that number lumps a repeated heading in with a leaked API key, so it shouldn't be read as nine in ten skills being broken. The shape of the problems matters more than the count. Almost all of them are ordinary packaging faults. The single most common defect, in more than half the files, was a skill that never says when it should be used. It describes what it does and stops there.
That's the one thing the file can't afford to leave out, because the description is the main basis on which an agent decides whether to load the skill at all. Routing is also the only part of this the researchers put in front of a functional test, and the test was deliberately crude. A lexical search engine, with no model in the loop, picked a skill for each task. Skills with clean routing metadata came top of the list 88.5% of the time, against 82.6% for the ones missing trigger guidance. A real gap, and a narrower one than the subject invites. The authors warn that a production agent selecting semantically may compress it further. What the study supports is that routing metadata is the clearest functional bottleneck they could measure. Not that most published skills sit there inert.
"The most direct reuse failure is that an otherwise useful skill is never selected."
This is a failure with no symptom. A broken skill that runs produces bad output someone notices. A skill that never gets selected produces nothing at all, not even an error, a log line or a complaint. The library keeps growing and nobody can say which half of it is doing any work. Any team with an internal skill library should be able to answer one question. Which skills actually got loaded last week? Most can't, because nothing anywhere is recording it.
The second finding cuts against the instinct to fix this by writing more. Problems pile up steeply with length, and the shortest skills were the cleanest by a wide margin. Among the complaints filed against real agent platforms was one installed set that added over four minutes to every cold start before the agent had done anything. Their exemplars were the 419 defect-free skills found in repositories with ten thousand stars or more, a measure of the repository's popularity rather than the skill's use. When the researchers looked at what those had in common, the one trait that wasn't circular was that each encoded something local, such as this repository's conventions or this service's deployment steps. A skill explaining how to write a good unit test is one your agent will dutifully load and learn nothing from.
"A reusable skill is not merely a saved prompt; it is a routed, progressively disclosed, executable capability."
Their explanation for all this, offered as a hunch rather than a result, is that most public skills are chat transcripts saved to disk. Someone solved a problem interactively, thought that was useful, and committed it. The same caution applies to the associations they report. Files stamped as AI-generated carried roughly twice the rate of safety problems, but authors who bother to stamp a file that way aren't a random sample of anyone.
Where this lands for a working team is how to spend review attention. Their automated repair pass fixed 58% of detected defects, and fixed none at all in the two safety categories covering injection and instructions that bypass a guard rail. Not a poor rate on those two, but zero. Both rest on small subsamples, and it isn't a zero across every safety defect the paper defines. It is still a decent argument for letting a linter own formatting and keeping human eyes on anything touching credentials or destructive commands.
Two things to hold on to before generalising any of it. The corpus is public and lopsided. The top five repositories supplied 23.5% of the files and nearly half of all repositories contributed exactly one skill, so this describes what people publish on GitHub rather than what a company's internal library looks like. And of the seven defect categories, only routing got that functional test. For the other six the paper measures how often something is malformed, not what being malformed costs.
"Defect-free status is not the goal."
Sources
- Zhang, C., Liu, Y., Chen, X. et al., "What Keeps Agent Skills from Being Reusable? Evidence from 138K SKILL.md Files" — https://arxiv.org/abs/2608.08453