When an agent scaffolds a new project, somebody picks the language. Increasingly that somebody is the agent, and nobody writes the decision down. Researchers at King's College London assembled a set of realistic projects, all of them places where Python is a poor fit, such as embedded firmware, a mobile app, a low-latency trading platform or a browser extension. Then they asked two dozen production models to build each one, without saying what to build it in.
Python turned up in 35% of the implementations. The same models, asked what language they'd recommend for the same projects, named Python only 11% of the time. Every model tested implemented in Python more often than it recommended it. The advice is roughly sound; the code doesn't follow it. Fewer than half of all implementations used any of the model's own top three recommendations.
The reasoning traces explain why, and the answer isn't bad judgment. It's no judgment. Fifteen of the twenty-five models expose their raw traces, and among the Python implementations those fifteen produced, the model never weighed the language at all in seven cases out of ten. It started solving the problem in Python the way a developer starts typing in whatever editor is already open. Most of the rest picked Python explicitly for being easy. The reading of the traces is an automated judge's, checked against human labels on a held-out sample of 150.
"In effect, the chain-of-thought starts too late: models reason about how to implement a chosen stack without first reasoning about which stack should be chosen."
Then there's the part that should bother anyone treating reasoning traces as an audit trail. Nearly one Python build in ten, across that same set of traces, either invented context to justify Python or announced a different language and wrote Python regardless. One model reasoned, "Wait, the original code example was in Python, so maybe they want Python?" There was no original code example. Every prompt was a single turn with no history at all.
"These models do not merely default to Python; they fabricate support for doing so, such as claiming that a previous response used Python or that the prompt required it, despite every experiment being single-turn and language-neutral."
Two things follow for anyone running agents on greenfield work. Asking and building are separate modes. A conversation in which the agent gives sound advice is not evidence about the code it writes ten minutes later. Strictly, what the study demonstrates is how heavily the answer depends on which prompt you use. It ran recommendation and implementation as separate single-turn prompts, and never tested a model asked to implement a choice it had just made. That handover is the experiment a team would want before trusting it. Either way, if the stack matters it has to be decided by the team and stated in the prompt, not inferred from an earlier chat with the same model.
The other is about where review can reach. Code review starts after the language is chosen. No diff shows the alternative that was never considered, and a language is among the hardest things to walk back once a codebase exists. Where an agent scaffolds a greenfield project for itself, that decision has moved out of the reviewed part of the process entirely.
"Until these failures are fixed, language and ecosystem choices should not be left implicit in agentic development."
This can be over-read. The benchmark stacked the deck on purpose, so the 35% is not a general error rate for agents. The two experienced developers who rated the tasks disagreed with each other on a quarter of them about how bad a fit Python really was. The researchers also never tested whether simply telling a model to weigh the language first fixes it. That probably helps. It isn't proven. Nor does the study look at whether any of the code it counted actually worked. This measures the choice of language, not the quality of what got built in it.
One finding does travel, and it applies directly the next time a vendor pitches a coding-specialised model. Tuning a model for code didn't cure the habit, and the code-tuned editions were often the Python-heaviest in the study. One frontier model's code edition reached for Python six percentage points more often than the general model it was built from. That is an association across a handful of families rather than an isolated effect of code tuning, but it points the opposite way to the pitch.
Sources
- Twist, L., Stone, T., Yannakoudakis, H. et al., "LangChoiceBench: Measuring and Explaining Programming-Language Choice in LLMs" — https://arxiv.org/abs/2608.06041