Most of the argument about how coding agents should reach their tools has run on a single number, the claim that agents talking to services over the Model Context Protocol burn around 35 times more tokens than agents using ordinary command-line programs. The figure comes from a practitioner blog post. It has been repeated in vendor decks ever since, and nobody had tried to reproduce it under controlled conditions.

A group of university researchers did. They fixed one task (find an open issue, create a branch, apply a patch, commit it, open a pull request, report a file count) and ran it across a range of popular agent harnesses and models, with and without an MCP server attached. Then they checked the repository to see whether the work had been done, rather than taking the agent's word for it.

They did not settle the argument. Within harnesses supporting both routes, MCP was sometimes far more expensive and sometimes cheaper, with no consistent direction. The number everyone has been quoting doesn't describe a stable property of anything.

"The quantity is not stable enough to have a single value."

The stable effect sat somewhere else. Holding the task, the models and the verification identical and changing only the harness driving them, the median input tokens a completed run consumed moved by a factor of twenty. The wrapper around the model, not the protocol it speaks, is the biggest thing moving that number. The cheapest harnesses in the study ship no MCP support at all. They did the whole job with a shell and the GitHub command-line client, and were among the most reliable of the lot.

Two things that headline doesn't carry. The rows of that table don't share a model mix, which is why the authors tell readers to compare them in tokens; every dollar figure in the paper is modelled from public list prices rather than read off an invoice. And tokens are not the whole bill. The same study finds an elevenfold spread in cost across the five models on near-identical token counts, arising almost entirely from price per token. Harness overhead is the variable nobody measures. It isn't the only one there is.

A small open-weight model running on one workstation finished the task under all seven harnesses, and in ten of the twelve harness-and-interface configurations it was given, and both misses completed sometimes when the runs were repeated. What changed was not whether the work got done but what it cost, with 17,000 input tokens under the leanest harness against 2.4 million under the most elaborate, the same model on the same job, a factor of 139.

"The model was never the constraint; the software around it was."

That reframes a decision plenty of teams are sitting on. Anyone waiting for small models to get good enough to self-host is waiting on the wrong variable. For work of this size they were already good enough, and the cost was being set downstream, by general-purpose tooling hauling machinery the job never used.

Agents also could not be relied on to use the interface they were handed. Given both routes to the repository, fewer than a third stuck to the one they had been assigned. Others did the whole job in the shell with an unused catalogue sitting in context, and a few bypassed both and called GitHub's web API directly. Adding an instruction to the prompt naming the right interface changed nothing. Removing the credentials for the wrong one did.

"A measurement that assigns an interface without verifying which interface was used reports the cost of an unknown mixture."

That is a controls problem. An agent that reaches a service by whichever path happens to be open can't be governed by a document describing the intended path. Configuration is the enforcement mechanism; the prompt is a suggestion.

The practical takeaway is narrower than the headline. A team should measure what its own harness costs on a task it runs repeatedly, before spending any energy on the protocol question. That number varies more than any protocol choice, and almost nobody looks at it. When a vendor quotes token efficiency, ask which harness produced the figure, how much of the input was cached, how many tool schemas ride along with every request, and whether anyone confirmed the work got finished. Failed runs burned more than successful ones and delivered nothing.

This will get over-read, because it's one task against GitHub, a service with an unusually good command-line client, and it says little about services offering only an MCP server. The lean harnesses also differ from the popular ones in more ways than MCP support, which the authors are careful to state. And nearly every configuration was run once. When the researchers repeated the locally served ones, identical setups varied by a typical factor of 1.5. That is the study's resolution limit, and a reason to treat the small gaps between named products as noise rather than as a ranking. The claim the data supports is narrower than "MCP is waste". The industry has been arguing loudly about a variable that moves the bill less than one it never examines.

Sources

  1. Alier Forment, M., Casañ Guerrero, M. J., García-Peñalvo, F. J., Pereira, J., "The Scaffolding Matters More Than the Interface: A Controlled Comparison of MCP and CLI Tool Use Across Seven Agent Scaffoldings, Five Language Models, and One Software Task" — https://arxiv.org/abs/2608.08654