Cache Residency Just Became the Real Price of an Agent

One production dashboard this week: $1,051 in DeepSeek V4 Pro spend against $3,351 in cache savings. A 3.2× effective discount that shows up nowhere on a rate card, invisible to every vendor comparison spreadsheet built in the last eighteen months.

The mechanism is boring and load-bearing. DeepSeek's disk-backed KV cache persists for hours. The industry norm is about five minutes. Agentic workloads spend most of their tokens inside loops — retries, tool calls, multi-turn reasoning against a stable system prompt — which is exactly the shape KV cache reuse eats for breakfast. On a twelve-step agent, cache residency is the line item. On short chats, rounding error. Most cost models were built for chats.

Meanwhile the leaderboard did what leaderboards do. Three open-weight MoE models — DeepSeek V4 Pro, Kimi K2.6, MiMo V2.5 Pro — landed at 52–54 on the Artificial Analysis Intelligence Index this week. GPT-5.5 sits at 60, Opus 4.7 and Gemini 3.1 Pro at 57. The gap that justified frontier API pricing is now smaller than the gap between a well-tuned harness and a sloppy one. And the remaining points live in hard-reasoning frontiers — HLE, CritPt, TerminalBench Hard — not in the multi-turn coding and tool use that most agent workloads actually do.

Grok 4.3 illustrates the second trap. Headline pricing dropped 40–60%: $1.25/M input, $2.50/M output. Then xAI added a $0.05 fee per safety-filter-blocked request. At a 2–3% refusal rate on production prompts, that quietly erodes the savings. The model also scored #1 on CaseLaw, 11% on ProofBench, and gained +321 Elo on GDPval-AA while regressing on Vending-Bench 2 badly enough that the agent reportedly preferred to sleep. Same weights, radically different verdicts depending on which harness ran the eval.

Yes, but — a fair reading is that closed-frontier labs will widen the gap on the next training run, as they have three times before, and the ninety-percent number will age like most ninety-percent numbers do. Probably true for the hardest reasoning tasks. Not true for the coding, tool-use, and multi-step planning that dominate production spend. The commodity floor moved. It won't move back.

The cost model most teams are running is broken in a specific way

Per-token pricing assumes each token costs the same. For agents it doesn't. A token inside a warm cache prefix is free. A token that forced a re-prefill because the provider evicted at the five-minute mark cost you a full prompt evaluation. The corrected model has three components: KV cache priced in GB-hours at the accelerator memory rate, active parameters priced in FLOPs per token, and shard footprint priced by the minimum deployable instance that total parameter count forces. MoE breaks the single-FLOPs figure most teams still use, because active parameters set the matmul bill and total parameters set the memory floor, and those diverge by 30× on a 1.6T/49B model.

Run a real agent trace through that model, not a synthetic benchmark. The ranking of which models are cheapest for your workload will change. Not by a little.

The harness is the confound

GPT-5.5 beats Opus 4.7 on the Intelligence Index and loses to it inside Claude Code's harness. Same weights, different plumbing. Hugging Face's Delangue calls this the engine-versus-car problem, which is generous — closed APIs bundle routing, tool schemas, retry logic, and sometimes multiple models behind one URL. Benchmarking an open-weight model through a harness tuned to Claude's tool-use conventions produces degradation that looks like a model problem and is an instrumentation problem.

The fix is not exotic. Log harness config as an eval dimension: prompt format, tool schema shape, retry policy, context budget. Re-run top candidates with per-family tuned harnesses. Any procurement checklist that accepts a benchmark score without the harness definition is accepting an unauditable claim. Most of them do.

What to do this week

Instrument three metrics as first-class variables in your agent runtime before the sprint ends: cache-hit rate, prefix-reuse ratio, effective $/1K tokens net of cache discounts. Most observability stacks track tokens and latency and miss cache state entirely. You cannot optimize what you cannot see, and the number you cannot see is roughly three times your visible bill.

Then replay last month's agent traffic through DeepSeek V4 Pro and Grok 4.3 in shadow mode. Compare blended $/successful-task, not sticker $/M tokens. If your current stack is stable-prefix-friendly, DeepSeek's cache TTL wins by a margin no per-token comparison would predict. If it isn't, that's the second finding — your system prompts and tool schemas are drifting between calls and killing cache reuse, and no model switch will fix a harness problem.

One more thing. Every SKILL.md file executing bash in your agent's environment is remote code execution by design. The Planner/Executor split roughly doubles inference cost and is the only deterministic defense against prompt injection when the agent touches untrusted content. Both are worth their tax on any path that reads the open web and writes anywhere real. The 3.2× cache discount pays for it and then some. That's the trade this week.