Prompt caching is a discount most teams running coding agents already take. The provider stores the work done on the long, unchanging opening of a prompt, the system instructions and tool schemas, then charges a fraction of the price the next time that opening appears. Agent harnesses supply it by design, and every public Claude Code, Cursor and OpenCode preamble the researchers collected was long enough to qualify before a single private word appeared.
The cache, though, lives on the provider's side of the gateway. A gateway authenticates each customer with its own key, then forwards everything upstream through one shared provider account, and the provider scopes its cache to that account. Across five widely used open-source gateways, not one kept customers in separate cache domains by default.
"Across five open-source gateways connected to OpenAI and Anthropic, none bound customers to upstream credentials by default; under a shared credential, all five exposed cross-customer cache reads for both providers."
OpenAI draws its cache boundary at the organisation and doesn't sell extra organisations on demand; Anthropic draws it one level finer but caps an account at a hundred. The authors' argument is that a gateway with more customers than the provider sells boundaries has to pool some of them, making this arithmetic rather than misconfiguration. Three of the five also pick a fresh credential per request, so adding accounts scatters a customer's content across more cache domains, not fewer.
"Counting API keys differs from counting security domains."
What that buys an outsider is narrower than it sounds. Mostly it confirms membership. Someone already holding your system prompt or a context document can establish it's in live use inside the pool, without learning whose. On one production route the team chained the signal further and recovered a short secret they had planted themselves, and they are careful about what that proves.
"We found no stable production setting for general extraction of arbitrary content."
A gateway with separate keys, separate billing and separate dashboards can still be one security domain, with nothing on the customer's side to show it. That changes what to ask a vendor sitting between your engineers and a model provider. "Does each team get its own key" describes authentication and settles nothing. Ask which upstream account your requests land on, and who else lands on it. A vendor who hears those as the same question hasn't checked.
The paper also prices the fix. Splitting the cache after the shared public preamble kept nearly all the discount for about two percent more; separating everything cost several times the unsafe baseline. Cheap enough to be an easy call, if anyone owned it. Mostly nobody does, because the setting sits in a gateway engineering treats as plumbing and procurement treats as a line item.
The study measured which paths can leak, not how often they do. Its production sample ranks model names by traffic rather than sampling real requests, and runs through one pair of accounts, so it can't say what share of live traffic sits in a shared domain today. That distinction is the first thing that will be lost when this gets repeated as a breach. The useful response is an inventory. Find every hop between your agents and the model and ask what identity it forwards. Or plant a unique string in your own preamble and watch whether it comes back warm somewhere it shouldn't.
Sources
- Sun, B., Cai, Y., Liu, X. et al., "KeyPooling: Measuring Where LLM API Relay Paths Collapse Prompt Cache Isolation" — https://arxiv.org/abs/2608.17485