When API costs cross the self-hosting line — the economics that drive AI infrastructure decisions, and how to spot the crossover before it hits your bill.
4 concepts 4 decision paths Calculator
Share
The economics explained
Click each model to see the cost dynamics and visual explanation.
Pay-per-token
API Economics
"A taxi — cheap for short trips, ruinous for a daily commute"
API pricing is elegantly simple: you pay per token in, per token out. At low volumes, this is unbeatable — no infrastructure, no GPUs, no ops team. But the cost scales linearly with volume. Double your queries, double your bill. There's no economy of scale, no volume discount that matters at 100K+ queries per day.
Take one query and hold it fixed: 4,000 tokens of context in, 600 out. At $2.50 per million input tokens and $10 per million output, that's $0.016 a query. At 1,000 queries/day it's $480/month — manageable. At 100,000 queries/day it's $48,000/month for the same query. At that point, you're not paying for AI — you're paying rent on someone else's GPUs.
Pay-per-token
API Economics
"A taxi — cheap for short trips, ruinous for a daily commute"
API pricing is elegantly simple: you pay per token in, per token out. At low volumes, this is unbeatable — no infrastructure, no GPUs, no ops team. But the cost scales linearly with volume. Double your queries, double your bill. There's no economy of scale, no volume discount that matters at 100K+ queries per day.
Take one query and hold it fixed: 4,000 tokens of context in, 600 out. At $2.50 per million input tokens and $10 per million output, that's $0.016 a query. At 1,000 queries/day it's $480/month — manageable. At 100,000 queries/day it's $48,000/month for the same query. At that point, you're not paying for AI — you're paying rent on someone else's GPUs.
Own the hardware
Self-Hosted Economics
"Buying a car — expensive upfront, but the commute is nearly free"
Self-hosting means GPU servers: A100s, H100s, or their cloud equivalents. Matching frontier quality means a 70B-class open-weight model, which means four H100-class GPUs at around $2,250 each per month plus roughly $3,000 of platform engineering — $12,000/month, serving about 40,000 of those 4,000-in / 600-out queries a day. That is the low end of the $10K-30K a capable cluster costs. Once you're paying it, the marginal cost per query is nearly zero: your 40,000th query costs the same as your first.
Capacity comes in chunks. The second chunk takes you to $21K/month and ~80K queries/day, the third to $30K and ~120K. Drop the quality bar and the numbers change completely: a 7B model on a single A10G-class GPU handles roughly 20K of these queries a day for about $700/month of hardware — call it 40K with 4-bit weights — against $9,600 for the same 20K through a frontier API. That is not like-for-like — a 7B model is not a frontier model — and that gap is what the hybrid pattern exists to harvest.
The inflection point
The Crossover Point
"The moment buying a car becomes cheaper than taxis"
There's a specific volume where the lines cross: API cost rising linearly meets self-hosted cost sitting flat. Below the crossover, API wins on simplicity. Above it, self-hosting wins on economics. It's arithmetic, not judgement — the fully-loaded self-hosted cost divided by the per-query API price. $12,000 ÷ $0.016 is 750,000 queries a month, or about 25,000 a day. A step function can cross a straight line more than once, though: the four-GPU chunk runs out at 40K/day, where the API is at $19,200 and the next four GPUs cost $21,000, so the API is briefly cheaper again up to about 43,750/day. That window is the only one, and it matters when you're sizing right at a boundary.
Change the query shape and the crossover moves with it. A long agentic call — 8,000 tokens in, 2,000 out — costs $0.040 and crosses the same $12,000 floor at 10,000 queries/day. A short classification query — 1,600 in, 400 out — costs $0.008 and doesn't cross until 50,000/day. That range is where the 10K-50K band comes from: one fixed cost divided by the per-query prices real workloads run at. So don't guess — price your own query shape, then benchmark it for a week on cost, latency and quality.
Best of both
Hybrid Architecture
"Own a car for the commute, take a taxi to the airport"
The pragmatic answer is rarely pure API or pure self-hosted. Route by complexity: simple, high-volume queries go to a cheap self-hosted SLM. Complex, nuanced queries that need frontier reasoning go to the API. You get most of the saving without lowering the quality ceiling.
A typical split: 80-90% of queries handled by a 7B-14B self-hosted model at near-zero marginal cost, the rest routed to a frontier API. At 100,000 queries/day that's 85,000 on five A10G-class GPUs at ~$700 each plus the $3,000 ops floor ($6,500), and 15,000 a day still on the API ($7,200) — $13,700/month against $48,000 for pure API and $30,000 for a frontier-quality cluster of your own. Roughly 70% off the API bill, with no quality compromise where it matters.
The cost ladder
One workload, priced five ways — 100,000 queries a day, 4,000 tokens in and 600 out.
Infrastructure choices at 100K queries/day
Each rung is the same traffic on different infrastructure, at $2.50 / $10 per million tokens
Frontier API, every query
No infrastructure, no ops
$48K/mo
Self-hosted 70B cluster
12 GPUs plus the ops floor
$30K/mo
Hybrid: SLM + frontier API
85% small model, 15% frontier
$13.7K/mo
Self-hosted SLM
7B-14B on 5 GPUs at 20K/day each, plus ops
$6.5K/mo
Self-hosted SLM, quantised
4-bit weights, 3 GPUs at 40K/day each, plus ops
$5.1K/mo
Halve the query length and every self-hosted rung holds while the API rung halves — which is why the crossover moves with query shape, not only with volume.
An AI cost curve plots the total cost of serving inference against request volume for each infrastructure option — per-token API, self-hosted GPU, and hybrid. API pricing is a straight line through the origin (you pay per request, forever); self-hosting is a high fixed cost plus a near-flat marginal cost. Comparing the curves shows where one option overtakes another as volume grows.
What is the crossover point, and how do I find it?
The crossover point is the request volume where a fixed-cost self-hosted setup becomes cheaper than per-token API pricing. Find it by dividing the fully-loaded monthly cost of the self-hosted option (GPU, hosting, and the engineering time to run it) by the per-request API price. Below that volume, APIs win on total cost; above it, self-hosting does — assuming you can keep the GPU busy.
At what volume does self-hosting beat API pricing?
There is no universal number — it depends on your per-request API price, model size, and how fully you utilise the hardware. As a rule of thumb: under ~1K requests/day, APIs almost always win because engineering overhead dwarfs token spend; between 1K and 50K/day is the benchmark zone; above ~50K/day the economics usually favour self-hosting the bulk of traffic. Always validate with a real benchmark on your own prompts.
Does self-hosting include the cost of running it?
It must, or the comparison is wrong. The honest self-hosted figure is GPU or instance cost plus storage and networking plus the engineering time to deploy, monitor, patch, and keep the model current. Teams that only count the GPU bill systematically overstate the savings and cross over later than the spreadsheet predicts.
Why does a hybrid approach cut costs so much?
Most workloads are long-tailed: the majority of requests are simple and a small fraction need frontier reasoning. Routing the bulk (often 85%) to a cheap self-hosted small model and reserving a frontier API for the hard 15% captures most of the self-hosting savings while keeping quality where it matters — commonly around a 70% reduction versus sending everything to a frontier API.
