FinOps for AI
AI spend scales with autonomy, not traffic. The circuit breakers, tier routing, caching, and chargeback discipline that keep LLM and agent spend answerable to somebody.
AI spend scales with autonomy, not traffic
An agent differs from a chatbot in one financially important way: it acts in a loop — perceive, reason, act, repeat — until it decides the task is finished. Most model APIs bill the full conversation history on every call, so each step of a loop resends everything that came before it. An agent that takes thirty steps doesn't cost three times one that takes ten — it costs closer to nine times, because the context grows with every pass.
Give an agent an API key and broad instructions and you've handed an intern a corporate credit card with "do whatever you think is best" written on the envelope. The intern isn't malicious — they're diligent, uncapped, and unsupervised. Traffic unchanged, users unchanged, spend tripled: the culprit is almost always the freedom you've given the software.
AI spend scales with autonomy, not traffic
An agent differs from a chatbot in one financially important way: it acts in a loop — perceive, reason, act, repeat — until it decides the task is finished. Most model APIs bill the full conversation history on every call, so each step of a loop resends everything that came before it. An agent that takes thirty steps doesn't cost three times one that takes ten — it costs closer to nine times, because the context grows with every pass.
Give an agent an API key and broad instructions and you've handed an intern a corporate credit card with "do whatever you think is best" written on the envelope. The intern isn't malicious — they're diligent, uncapped, and unsupervised. Traffic unchanged, users unchanged, spend tripled: the culprit is almost always the freedom you've given the software.
Three circuit breakers every agent needs
None of this requires exotic engineering — it's the discipline distributed systems learned decades ago, applied to a component that spends money instead of threads. A hard step cap catches wandering. A repetition check catches oscillation — an agent bouncing between two states stays comfortably under any generous cap while burning money the entire time. A session budget that actually kills the credential catches everything else.
The test to apply: if this agent's prompt were subtly broken tomorrow, what is the maximum it could spend before a human found out? If the honest answer is "whatever's on the card", you don't have a cost model — you have exposure. Meter the budget at a gateway, never inside the agent itself: asking the runaway process to police itself is not a control.
Seeing the bill isn't governing it
Measurement is not governance. Knowing you spent $100,000 with a model provider last month is trivia unless you also know which team spent it, on which use case, and which of those would survive a business-case review. The maturity path runs from basic showback (request counts by team), through advanced showback (tokens and cost), to entitlement enforcement (limits applied at the gateway before the overrun), and finally chargeback (usage drives real internal invoices).
Most organisations are at level one and believe they're at three. The tell is a simple question: who gets paged when a team blows through its AI budget? If the answer is "nobody, we review it monthly", the circuit breakers have no owner — and an unowned control is a decoration. Attribution is what makes savings durable, because attribution creates owners, and owners notice drift.
Run the calculator FinOps for AI: Cost Governance Calculator
Model your AI spend across tier routing, semantic caching, and agentic budget controls — find where governance saves money before the invoice arrives.
Decision framework
I'm speaking on this — The Compute Infrastructure Questions Every AI Buyer Should Ask →