What an AI Agent Actually Costs in Tokens: A Reproducible Teardown

Teams that get a shocking agent bill usually blame the model's price. They switch to a cheaper model, save a little, and stay confused about why the bill is still high. The price was never the main problem. The way the agent uses context is.

I've spent 30 years building production systems, and the token economics of an agent follow a pattern that's easy to miss in a demo and impossible to ignore at scale. This is a teardown of where the cost actually goes, with a model you can run on your own numbers rather than take on faith.

Why the bill grows faster than you expect

A single model call is cheap. An agent is not a single call. It's a loop: the model thinks, calls a tool, reads the result, thinks again. The trap is what it carries between steps.

A naive agent re-sends the whole conversation plus every past tool output on every step. So step 2 pays to re-read step 1. Step 5 re-reads steps 1 through 4. By step 10 it's re-reading everything that came before, again. The input tokens don't grow with the number of steps. They grow with the square of it.

That curve is the whole story. The gap between the two lines is money, and it widens with every step you add.

A cost model you can reproduce

Here's the model behind the chart. Take a 10-step agent with a 2,000-token system prompt, and assume each step adds roughly 1,000 tokens of model reasoning and produces a 2,000-token tool output. A naive agent's total input tokens across the run come to:

naive_input ≈ N·S + (R+T)·N·(N−1)/2

where N is the number of steps, S the system prompt, R the reasoning tokens per step, and T the tool output per step. With N=10, S=2000, R+T=3000, that's 20,000 + 3,000 × 45 = 155,000 input tokens for one run. The N·(N−1)/2 term is the quadratic part, and it dominates everything else.

Now apply two changes. Keep a running summary of old turns instead of the full history (about 500 tokens), and store each tool output and pass a short reference to it rather than the full 2,000 tokens. Per-step input drops to roughly system + summary + current step ≈ 2,000 + 500 + 3,000 = 5,500, which is linear in the number of steps. Over 10 steps that's about 54,500 input tokens, down from 155,000. A 65% cut, and the model never changed.

	Naive agent	Optimized agent
Context per step	Full history + all tool outputs	Running summary + current step
Past tool outputs	Re-sent in full every step	Stored, passed by reference
Growth in step count	Quadratic	Linear
Input tokens (10-step model)	~155,000	~54,500

At an example rate of $3 per million input tokens, that's about $0.47 versus $0.16 per run. Run the agent 100,000 times a month and the modeled difference is roughly $47,000 versus $16,000. Substitute your model's current price and your own token counts; the percentage holds regardless of the rate, because it comes from the architecture.

The five changes, ranked by how much they move the bill

Not every technique matters equally. Ranked by impact on a typical multi-step agent:

Technique	What it does	Why it helps
Context compaction	Replace old turns with a running summary	Kills the quadratic term. Biggest single lever.
Tool-result externalization	Store large outputs, pass a short reference	Stops re-sending the heaviest payloads each step
Prompt caching	Cache the static system prompt and tool schema	Cuts the cost of the fixed part you send every call
Model routing	Send simple steps to a cheaper model	Lowers per-token price on the easy work
Structured outputs	Constrain the model to a schema	Trims output tokens and downstream parsing waste

Compaction and externalization are where most of the savings live, because they attack the part of the cost that compounds. Caching and routing are real but secondary. Reach for them after you've flattened the curve, not before.

Where a cheaper model actually helps

Routing has a place. Some steps in an agent are trivial: a yes/no classification, a short extraction, a format check. Those don't need your most expensive model, and sending them to a small one is free money. But routing lowers the price per token; it does nothing about how many tokens you resend. If the underlying pattern is quadratic, a cheaper model just gives you a slightly cheaper quadratic. Flatten the curve first, then route what's left.

Run it on your own numbers

The formula above is the point. Plug in your real step count, system-prompt size, and typical tool-output size, and you'll see where your own bill is going before you change a line of code. If your tool outputs are large (retrieved documents, API dumps, transcripts), externalization will dominate your savings. If your agents run long, compaction will. The model tells you which lever to pull, instead of guessing.

These figures are modeled from the stated assumptions, not measured from one specific system, so treat the dollar amounts as an example and the percentages as the durable part. The quadratic is real, and it's the reason agent bills surprise people. Once you can see it, it stops being a surprise and starts being a design decision.

---

Related: Deterministic AI agents and the gatekeeper architecture · Why 95% of enterprise AI pilots die.

Vikas Goel is the founder of Thinkerwave AITech and a former enterprise CTO. Over 30 years he has built production systems and shipped enterprise voice AI used by millions. He works with enterprises and their India GCCs as a fractional AI CTO and AI advisor.