AI spend scales with autonomy, not traffic. The circuit breakers, tier routing, caching, and chargeback discipline that keep LLM and agent spend answerable to somebody.
3 concepts 4 decision paths Calculator
Share
Where AI spend goes wrong
AI spend scales with autonomy, not traffic
An agent differs from a chatbot in one financially important way: it acts in a loop — perceive, reason, act, repeat — until it decides the task is finished. Most model APIs bill the full conversation history on every call, so each step of a loop resends everything that came before it. An agent that takes thirty steps doesn't cost three times one that takes ten — it costs closer to nine times, because the context grows with every pass.
Give an agent an API key and broad instructions and you've handed an intern a corporate credit card with "do whatever you think is best" written on the envelope. The intern isn't malicious — they're diligent, uncapped, and unsupervised. Traffic unchanged, users unchanged, spend tripled: the culprit is almost always the freedom you've given the software.
AI spend scales with autonomy, not traffic
An agent differs from a chatbot in one financially important way: it acts in a loop — perceive, reason, act, repeat — until it decides the task is finished. Most model APIs bill the full conversation history on every call, so each step of a loop resends everything that came before it. An agent that takes thirty steps doesn't cost three times one that takes ten — it costs closer to nine times, because the context grows with every pass.
Give an agent an API key and broad instructions and you've handed an intern a corporate credit card with "do whatever you think is best" written on the envelope. The intern isn't malicious — they're diligent, uncapped, and unsupervised. Traffic unchanged, users unchanged, spend tripled: the culprit is almost always the freedom you've given the software.
Three circuit breakers every agent needs
None of this requires exotic engineering — it's the discipline distributed systems learned decades ago, applied to a component that spends money instead of threads. A hard step cap catches wandering. A repetition check catches oscillation — an agent bouncing between two states stays comfortably under any generous cap while burning money the entire time. A session budget that actually kills the credential catches everything else.
The test to apply: if this agent's prompt were subtly broken tomorrow, what is the maximum it could spend before a human found out? If the honest answer is "whatever's on the card", you don't have a cost model — you have exposure. Meter the budget at a gateway, never inside the agent itself: asking the runaway process to police itself is not a control.
Seeing the bill isn't governing it
Measurement is not governance. Knowing you spent $100,000 with a model provider last month is trivia unless you also know which team spent it, on which use case, and which of those would survive a business-case review. The maturity path runs from basic showback (request counts by team), through advanced showback (tokens and cost), to entitlement enforcement (limits applied at the gateway before the overrun), and finally chargeback (usage drives real internal invoices).
Most organisations are at level one and believe they're at three. The tell is a simple question: who gets paged when a team blows through its AI budget? If the answer is "nobody, we review it monthly", the circuit breakers have no owner — and an unowned control is a decoration. Attribution is what makes savings durable, because attribution creates owners, and owners notice drift.
Why is AI spend different from ordinary cloud spend?
Because it scales with autonomy, not traffic. An agent acts in a loop until it decides the task is finished, and most model APIs bill the full conversation history on every call — so each step resends everything before it. An agent that takes thirty steps does not cost three times one that takes ten; it costs closer to nine times. Users unchanged, traffic unchanged, spend tripled.
What circuit breakers does every agent need?
Three, in order. A hard step cap, because an agent that has not converged by step ten or fifteen is looping rather than persevering. A repetition check — embedding similarity against recent steps — because an agent oscillating between two states stays comfortably under any generous cap while burning money the whole time. And a session token budget that actually kills the credential when spent. Meter all of it at a gateway, never inside the agent: asking a runaway process to police itself is not a control.
How much does model tier routing save?
Enough that it is a budget control rather than an architecture preference. Routing 80–90% of traffic to small, cheap tiers and reserving frontier reasoning for the remainder attacks the largest single line item in most AI estates. An estate without tiering has a cost floor set by its most expensive model, and no amount of dashboarding will lower it — using a frontier model to classify sentiment is couriering a Post-it note.
Does semantic caching actually work?
It works where traffic repeats, and the honest number is a spread rather than a headline. Serving a previously paid-for answer when a new query means the same thing can reach around 70% savings on support and FAQ flows, but lands closer to 10% on genuinely diverse traffic. Cache the workloads that repeat and do not build the budget on the ceiling. For RAG workloads, add context compaction — retrieval bloat commonly runs three to five times the tokens of a plain query.
What is the difference between seeing AI spend and governing it?
One question: who gets paged when a team blows through its AI budget? If the answer is "nobody, we review it monthly", you have showback, not governance. The path runs from basic showback through advanced showback to entitlement enforcement — limits applied at the gateway before the overrun — and finally chargeback, where usage drives real internal invoices. Attribution is what makes savings durable, because attribution creates owners, and owners notice drift.
When should I self-host a model rather than use an API?
When the workload is sufficiently predictable and sustained that fully loaded capacity plus operations is cheaper than the managed API path — and when the value of control, residency or latency justifies the operating responsibility. Compare unit cost at the required quality, then include utilisation, reliability, security, evaluation, on-call and transition cost. A cheap GPU list price is not a deployment model.
