Cheaper Tokens, Bigger Bills

Koko: Token prices are falling ninety-five percent by 2030. AI bills are tripling. Both of those things are true at the same time.

Sam: Which sounds like someone did the math wrong.

Koko: Right? And that's exactly the tension the piece is pulling on. Gartner is calling it the inference paradox — unit costs collapsing while total spend goes the other direction.

Sam: And the piece has a very specific explanation for how that happens, which is not just 'companies are buying more AI.' It's more mechanically interesting than that.

Koko: It is. We're going deep on one piece today — it's called Cheaper Tokens, Bigger Bills — and the whole argument turns on one thing: who runs the loop.

Sam: The loop being the key word that a lot of enterprise buyers are about to get very familiar with.

Koko: So let's set up the actual claim. The piece is making a structural argument: the reason your AI bill goes up even when the per-token price goes down is that you switched from a single model call — one prompt, one response — to a loop. And loops take many turns.

Sam: Walk me through the arithmetic they use, because it's elegant.

Koko: Cost per unit of work is roughly: number of turns, times tokens per turn, times price per token, plus everything else each turn drags along — retrieval calls, tool round trips, any human review on escalations.

Sam: So if you're doing a single prompt, turns is one. Fixed.

Koko: Exactly. But in an agent loop, turns is a variable — and here's the kicker — it's a variable the model itself sets at runtime. You don't control it ahead of time.

Sam: And the piece says loops routinely take ten or more turns on real work.

Koko: Right. So a ninety-five percent fall in the per-token price — which is huge — gets completely swamped by a change of one order of magnitude in the first factor. The number of turns.

Sam: That is a clean way to say it. The savings are real. They just show up in a completely different column than the costs do.

Koko: And that's the core of Cheaper Tokens, Bigger Bills. It's not that the efficiency gains aren't happening. It's that adoption is scaling faster than those gains.

Sam: The piece actually says that directly — total AI spend rises even as inference costs fall because adoption scales faster than efficiency gains. Which is a slightly uncomfortable thing to say out loud in a boardroom.

Koko: Especially when you're the one who approved the AI budget.

Koko: Let's go through the numbers, because the piece is notably careful about sourcing. Everything is dated August eighteenth, twenty-twenty-six.

Sam: Start with the Gartner one, because that's the one everyone is going to quote out of context.

Koko: Token costs fall roughly ninety-five percent by twenty-thirty, while agentic inference costs rise roughly five times over two years. That's what Gartner is calling the inference paradox.

Sam: Two years versus the back half of this decade. So the cost spike is now; the savings are later and projected.

Koko: That sequencing matters a lot for anyone building a business case this quarter.

Sam: What about the readiness numbers? Because those are striking.

Koko: IBM found that only eleven percent of technology executives say they're prepared for agent deployment at scale. And two-thirds of those same executives say they're accountable for AI systems they don't fully control.

Sam: That second number is the one I keep thinking about. Accountable but not in control. That's a governance problem with a very specific shape.

Koko: And it connects to the HFS and TCS number — only thirty-five percent of C-suite leaders say AI consistently delivers outcomes, earns regulator confidence, and allows control. All three at once.

Sam: So most leaders are getting one or two of those. Not the trifecta.

Koko: Now the one the piece says people misread — Anthropic's annualized revenue reaching sixty-five billion dollars at the end of July, up from nine billion eight months earlier.

Sam: That is a remarkable number. But the piece is clear: that is not evidence about your unit costs.

Koko: It's evidence that aggregate enterprise consumption is rising steeply while unit prices fall. It's the paradox stated from the seller's side of the invoice.

Sam: Which means the drop in per-token prices is real — Anthropic's customers are getting cheaper tokens — and the total bill is still going up because they're running a lot more loops.

Koko: The vendor's revenue growth and your cost efficiency can both be true simultaneously. They're just describing different things.

Sam: I feel like that sentence should be on a slide somewhere.

Koko: Here's where I want to push on something, because the piece makes a move that I think is underappreciated. It spends real time arguing that most of what is sold as agentic isn't actually an agent.

Sam: The control path test.

Koko: Right. The discriminator is: at runtime, what decides the next step — coded logic, or the model? And the piece lays out a six-rung ladder. Rungs one through four, the path is predefined. That includes RAG, it includes LLM workflows with routers, it includes deterministic automation.

Sam: And only rungs five and six — bounded single agent and multi-agent systems — are actually agent territory.

Koko: The piece says most of what's sold as agentic sits at rung three or four. Which is not a criticism of those products.

Sam: But it absolutely is a statement about what you're budgeting for, because a fixed path and an open loop have completely different cost shapes.

Koko: And this is where I'd push back a little on the framing — or at least complicate it. The piece advocates for minimum sufficient agency, the idea that the correct answer is the lowest rung that does the job.

Sam: Which I think is right, but it creates a slightly awkward dynamic where you're telling organizations: don't build the thing you've been sold on, build the less exciting thing.

Koko: The piece actually acknowledges this tension. It says productivity is currently accruing to a small vanguard of early adopters rather than to the median enterprise employee, and that broad workforce productivity figures are misleading for at least the next twelve months.

Sam: That's a rare piece of intellectual honesty. It's saying: here's the structural argument, here's the evidence, and here's what we're not claiming.

Koko: The narrower claim is that the loop is the unit that can be designed, controlled, observed, and priced — and that the organizations embedding control into their loops now are already ahead.

Sam: Already ahead of the ones waiting for a governance framework to mature. That's a pointed line.

Koko: And it's pointed because a lot of enterprises are doing exactly that — waiting.

Sam: Alright. Executive is listening, has two minutes before their next meeting. What do they actually do with this?

Koko: Three things, drawn directly from the piece. First: change your unit of measure. The piece is explicit — the right unit is cost per accepted outcome, not cost per token, not cost per call, not cost per seat.

Sam: Because token accounting flatters the cheaper-but-worse option.

Koko: Exactly. A loop that takes twenty turns and produces something a controller signs off on is cheaper than one that takes four turns and produces something that gets reworked by hand. Token counts alone will mislead you.

Sam: Second?

Koko: Treat observability as a finance control. The piece makes this argument clearly: an unobservable loop is an unbudgetable one. The invoice gives you one number — total monthly spend — with no decomposition.

Sam: And you can't compute a return if you don't have a complete cost picture per run.

Koko: The piece specifies exactly what a trace sufficient to budget from needs to carry: the run identifier, tools called and their outcomes, turn count, tokens in and out, escalations, and whether the result was accepted downstream. It calls this the agent equivalent of a general ledger entry.

Sam: Which is a frame that a CFO will immediately understand.

Koko: That's the point. Third: use the control path test before you buy anything. Before a vendor demo, ask — at runtime, what decides the next step? Coded logic or the model? If they can't answer that clearly, you're probably looking at rung three or four being sold as rung five.

Sam: And rung three with rung-five pricing is a very specific problem.

Koko: The piece gives you the vocabulary to have that conversation. Use it.

Sam: So to bring it back to the central tension — the title is doing a lot of work here. Cheaper Tokens, Bigger Bills. The whole essay is essentially an explanation of why both halves of that title are simultaneously true and will remain true for the foreseeable future.

Koko: And the resolution isn't 'wait for the tokens to get cheaper enough to cancel out the loop cost.' The resolution is: understand what you're running, measure what it actually costs per outcome, and build the control structures now rather than retrofitting them later.

Sam: The organizations that have a trace per run, that know their cost per accepted outcome, that can tell the difference between a rung-four workflow and a rung-five agent — those are the ones the piece says are already ahead.

Koko: And the forward-looking question I keep sitting with is this: as multi-agent systems become more common — rung six, where several goal-directed agents are coordinating — the turn count problem compounds. If one agent can take ten or more turns, what does a system of agents running in parallel look like on an invoice?

Sam: The answer is probably 'unrecognizable,' which is exactly why the observability argument isn't optional.

Koko: The full piece — Cheaper Tokens, Bigger Bills — is on Koko Knows, every figure source-linked and date-stamped. If a number moves, it records when and how. Worth reading once the way it's meant to be read: slowly, with the source links open.

Sam: And probably before your next AI vendor meeting.

Koko: Especially before your next AI vendor meeting.