FinOps for AI: Cost Governance Calculator

Input + output combined. Simple chat ≈ 1–2k. RAG ≈ 4–8k. Agentic ≈ 8–20k.

Blended in/out. Frontier APIs ≈ $8–45 depending on model and reasoning depth.

Frontier keeps whatever isn't routed below: 100% of queries.

Hosted open weights ≈ $0.40–2 per 1M blended.

Small language models ≈ $0.04–0.10 per 1M blended. Routing shares are capped at 100% combined.

% of queries semantically similar to a prior query. Support/FAQ ≈ 50–80%. Creative ≈ 5–15%. Mixed ≈ 25–35%.

Managed semantic caches typically hit 70–90% of similar queries. Infra: small ≈ $100/mo, large ≈ $2k+/mo.

Prompt compression, deduplication, and chunk-size tuning typically cut RAG context 20–50%.

Simple tasks ≈ 3–5 steps. Research pipelines ≈ 10–20. Autonomous coding ≈ 15–30.

Real-world agents incur 10–50% overhead from retries, reflection, and tool errors; runaway loops can exceed 200%.

A hard step cap kills wandering; the session budget kills everything else. Each unmanaged incident is modelled at ~5,000 steps before someone notices.