Glossary
Plain-English definitions of the AI infrastructure economics terms used across the strategy frameworks — cost curves, crossover points, hybrid inference, quantisation, and more.
- AI cost curve
-
A plot of the total cost of serving AI inference against request volume, drawn once per infrastructure option (per-token API, self-hosted GPU, hybrid). API pricing is a straight line through the origin; self-hosting is a high fixed cost plus a near-flat marginal cost. Comparing the curves reveals where each option is cheapest as volume grows.
See: AI Cost Curves
- Crossover point
-
The request volume at which a fixed-cost self-hosted setup becomes cheaper than per-token API pricing. Found by dividing the fully-loaded monthly cost of self-hosting (hardware plus the engineering time to run it) by the per-request API price. Below it, APIs win on total cost; above it, self-hosting does — provided the hardware stays busy.
See: AI Cost Curves
- Hybrid inference
-
An architecture that routes the bulk of requests to a cheap self-hosted small model and reserves a frontier API for the minority that need advanced reasoning. Because most workloads are long-tailed, hybrid routing captures the majority of self-hosting savings while preserving quality on the hard cases — commonly cutting spend by 80% or more versus an all-frontier approach.
See: AI Cost Curves
- Inference
-
Running a trained model to produce an output (a completion, classification, or embedding) in response to a request. Inference is the recurring, per-request cost of operating AI in production — as opposed to training, which is a one-off up-front cost — and is the cost that AI cost curves are drawn against.
- Per-token pricing
-
The usage-based billing model of hosted inference APIs, where you pay a fixed amount per thousand input and output tokens with no upfront commitment. It has zero fixed cost and a constant marginal cost, which makes it unbeatable at low volume and progressively expensive as request volume scales.
- Quantisation
-
Reducing the numerical precision of a model’s weights (for example from 16-bit to 4-bit) so it needs less memory and runs faster, usually with only a small quality loss. Quantisation is what makes self-hosted and edge inference economic: a quantised small model can run on a single GPU that a full-precision model would not fit.
See: Model Compression