# AI Cost Curves

Source: John Bradshaw — https://bradshaw.cloud
URL: https://bradshaw.cloud/strategy/ai-cost-curves/
Type: insight
Date: 2026-04-04
Updated: 2026-08-16
Tags: Economics, AI, Infrastructure

> When API costs cross the self-hosting line — the economics that drive AI infrastructure decisions, and how to spot the crossover before it hits your bill.

## The economics explained

Click each model to see the cost dynamics and visual explanation.

### API Economics
_Pay-per-token_

**Analogy:** A taxi — cheap for short trips, ruinous for a daily commute

API pricing is elegantly simple: you pay per token in, per token out. At low volumes, this is unbeatable — no infrastructure, no GPUs, no ops team. But the cost scales linearly with volume. Double your queries, double your bill. There's no economy of scale, no volume discount that matters at 100K+ queries per day.

**Business take:** Take one query and hold it fixed: 4,000 tokens of context in, 600 out. At $2.50 per million input tokens and $10 per million output, that's $0.016 a query. At 1,000 queries/day it's $480/month — manageable. At 100,000 queries/day it's $48,000/month for the same query. At that point, you're not paying for AI — you're paying rent on someone else's GPUs.

### Self-Hosted Economics
_Own the hardware_

**Analogy:** Buying a car — expensive upfront, but the commute is nearly free

Self-hosting means GPU servers: A100s, H100s, or their cloud equivalents. Matching frontier quality means a 70B-class open-weight model, which means four H100-class GPUs at around $2,250 each per month plus roughly $3,000 of platform engineering — $12,000/month, serving about 40,000 of those 4,000-in / 600-out queries a day. That is the low end of the $10K-30K a capable cluster costs. Once you're paying it, the marginal cost per query is nearly zero: your 40,000th query costs the same as your first.

**Business take:** Capacity comes in chunks. The second chunk takes you to $21K/month and ~80K queries/day, the third to $30K and ~120K. Drop the quality bar and the numbers change completely: a 7B model on a single A10G-class GPU handles roughly 20K of these queries a day for about $700/month of hardware — call it 40K with 4-bit weights — against $9,600 for the same 20K through a frontier API. That is not like-for-like — a 7B model is not a frontier model — and that gap is what the hybrid pattern exists to harvest.

### The Crossover Point
_The inflection point_

**Analogy:** The moment buying a car becomes cheaper than taxis

There's a specific volume where the lines cross: API cost rising linearly meets self-hosted cost sitting flat. Below the crossover, API wins on simplicity. Above it, self-hosting wins on economics. It's arithmetic, not judgement — the fully-loaded self-hosted cost divided by the per-query API price. $12,000 ÷ $0.016 is 750,000 queries a month, or about 25,000 a day. A step function can cross a straight line more than once, though: the four-GPU chunk runs out at 40K/day, where the API is at $19,200 and the next four GPUs cost $21,000, so the API is briefly cheaper again up to about 43,750/day. That window is the only one, and it matters when you're sizing right at a boundary.

**Business take:** Change the query shape and the crossover moves with it. A long agentic call — 8,000 tokens in, 2,000 out — costs $0.040 and crosses the same $12,000 floor at 10,000 queries/day. A short classification query — 1,600 in, 400 out — costs $0.008 and doesn't cross until 50,000/day. That range is where the 10K-50K band comes from: one fixed cost divided by the per-query prices real workloads run at. So don't guess — price your own query shape, then benchmark it for a week on cost, latency and quality.

### Hybrid Architecture
_Best of both_

**Analogy:** Own a car for the commute, take a taxi to the airport

The pragmatic answer is rarely pure API or pure self-hosted. Route by complexity: simple, high-volume queries go to a cheap self-hosted SLM. Complex, nuanced queries that need frontier reasoning go to the API. You get most of the saving without lowering the quality ceiling.

**Business take:** A typical split: 80-90% of queries handled by a 7B-14B self-hosted model at near-zero marginal cost, the rest routed to a frontier API. At 100,000 queries/day that's 85,000 on five A10G-class GPUs at ~$700 each plus the $3,000 ops floor ($6,500), and 15,000 a day still on the API ($7,200) — $13,700/month against $48,000 for pure API and $30,000 for a frontier-quality cluster of your own. Roughly 70% off the API bill, with no quality compromise where it matters.

## The cost ladder

One workload, priced five ways — 100,000 queries a day, 4,000 tokens in and 600 out.

### Infrastructure choices at 100K queries/day

Each rung is the same traffic on different infrastructure, at $2.50 / $10 per million tokens

- Frontier API, every query — No infrastructure, no ops — $48K/mo
- Self-hosted 70B cluster — 12 GPUs plus the ops floor — $30K/mo
- Hybrid: SLM + frontier API — 85% small model, 15% frontier — $13.7K/mo
- Self-hosted SLM — 7B-14B on 5 GPUs at 20K/day each, plus ops — $6.5K/mo
- Self-hosted SLM, quantised — 4-bit weights, 3 GPUs at 40K/day each, plus ops — $5.1K/mo

_Halve the query length and every self-hosted rung holds while the API rung halves — which is why the crossover moves with query shape, not only with volume._

## Decision framework

### Under 1K queries/day?

**API. The infrastructure overhead of self-hosting costs more than the tokens.**

At low volume, your engineering time managing GPUs, updates, and monitoring costs more than the per-token API spend. Keep it simple.

### Between 1K-50K queries/day?

**Benchmark. Run a 7B model on a single GPU for a week. Compare cost and quality.**

This is the crossover zone. Run a real benchmark with your actual prompts — synthetic benchmarks overstate quality and understate latency.

### Over 50K queries/day?

**Self-host the bulk. Use the API for the 10-15% of queries that need frontier reasoning.**

At scale, API pricing becomes your largest AI cost. A hybrid approach — SLM for bulk, frontier for hard queries — cuts spend by roughly 70% while maintaining quality.

### Latency-sensitive (edge inference)?

**Self-host with quantised models. API round-trips add 200-500ms you can't optimise away.**

Network round-trips to cloud APIs add 200-500ms minimum. For real-time UX, edge-deployed quantised models running on local GPUs are the only viable path.

## Frequently asked questions

### What is an AI cost curve?

An AI cost curve plots the total cost of serving inference against request volume for each infrastructure option — per-token API, self-hosted GPU, and hybrid. API pricing is a straight line through the origin (you pay per request, forever); self-hosting is a high fixed cost plus a near-flat marginal cost. Comparing the curves shows where one option overtakes another as volume grows.

### What is the crossover point, and how do I find it?

The crossover point is the request volume where a fixed-cost self-hosted setup becomes cheaper than per-token API pricing. Find it by dividing the fully-loaded monthly cost of the self-hosted option (GPU, hosting, and the engineering time to run it) by the per-request API price. Below that volume, APIs win on total cost; above it, self-hosting does — assuming you can keep the GPU busy.

### At what volume does self-hosting beat API pricing?

There is no universal number — it depends on your per-request API price, model size, and how fully you utilise the hardware. As a rule of thumb: under ~1K requests/day, APIs almost always win because engineering overhead dwarfs token spend; between 1K and 50K/day is the benchmark zone; above ~50K/day the economics usually favour self-hosting the bulk of traffic. Always validate with a real benchmark on your own prompts.

### Does self-hosting include the cost of running it?

It must, or the comparison is wrong. The honest self-hosted figure is GPU or instance cost plus storage and networking plus the engineering time to deploy, monitor, patch, and keep the model current. Teams that only count the GPU bill systematically overstate the savings and cross over later than the spreadsheet predicts.

### Why does a hybrid approach cut costs so much?

Most workloads are long-tailed: the majority of requests are simple and a small fraction need frontier reasoning. Routing the bulk (often 85%) to a cheap self-hosted small model and reserving a frontier API for the hard 15% captures most of the self-hosting savings while keeping quality where it matters — commonly around a 70% reduction versus sending everything to a frontier API.

## Diagrams

Standalone SVG diagrams, licensed CC BY 4.0 with attribution to John Bradshaw — link back to https://bradshaw.cloud/strategy/ai-cost-curves/ when embedding.

- [API Economics (SVG)](https://bradshaw.cloud/strategy/ai-cost-curves/diagrams/api.svg)
- [Self-Hosted Economics (SVG)](https://bradshaw.cloud/strategy/ai-cost-curves/diagrams/selfhost.svg)
- [The Crossover Point (SVG)](https://bradshaw.cloud/strategy/ai-cost-curves/diagrams/crossover.svg)
- [Hybrid Architecture (SVG)](https://bradshaw.cloud/strategy/ai-cost-curves/diagrams/hybrid.svg)

## Interactive companion

**AI Cost Curves: API vs Self-Host Crossover** — Find the request volume where self-hosted AI inference starts to beat per-token API pricing — an interactive calculator for AI cost crossover points.

Live calculator: https://bradshaw.cloud/strategy/ai-cost-curves/calculator/

## Related

- [Read the full article](https://bradshaw.cloud/writing/ai-cost-curves/) — when API pricing becomes the biggest infrastructure decision you make.
- [AI fundamentals](https://bradshaw.cloud/strategy/ai-fundamentals/) — the underlying concepts (RAG, fine-tuning, quantisation).
- [Frontier models](https://bradshaw.cloud/strategy/frontier-models/) — the model-choice side of the same economics.
- [Typed decision models](https://bradshaw.cloud/strategy/system-one-models/) — the cheap judgment layer that sits in front of the curve.
- [AI Grid](https://bradshaw.cloud/strategy/ai-grid/) — the infrastructure that makes edge inference economic.
- [Agentic AI patterns](https://bradshaw.cloud/strategy/agentic-ai/) — agents that need cheap inference at scale.
- [All strategy insights](https://bradshaw.cloud/strategy/)