LLM Self-Host vs API Cost Calculator (Breakeven Tokens)

Real self-hosting costs more than GPU rent: engineering time to deploy and keep an inference server healthy, on-call and monitoring, redundancy (a second node if you need failover), storage and egress, and the latency and quality trade-offs of quantization. None of those are in this number — the calculator isolates the raw compute economics, which is the right first filter but not the final answer. If the breakeven says self-hosting saves less than an engineer-week per month, the API is almost certainly the better deal once operations are priced in. This is a planning estimate, not procurement or financial advice.

The prefills come from NVIDIA's TensorRT-LLM performance overview and published third-party vLLM benchmarks, and they are order-of-magnitude estimates, not guarantees. Real throughput swings by several times with the serving engine, precision (FP8 vs BF16), batch size, and especially sequence lengths — the same 8B model on the same H100 does ~26,400 tok/s on short prompts but ~9,500 on long-context work. Benchmark your own workload before renting a fleet; the field is editable precisely so you can paste your measured number.

The breakeven formula answers the entry question: at what volume does the first rented GPU pay for itself against the API? Past that point, cost per token on the self-host side keeps falling until you cross into a second card, where it steps up again — the verdict at your entered volume accounts for that whole-GPU rounding. For models too big for one card (a 70B-class model typically needs 2–4 GPUs' worth of VRAM), enter the per-GPU share of your cluster's aggregate throughput, or check the GPU VRAM calculator first to size the deployment. Prices are a 2026-07 snapshot — verify both GPU rates and API prices before deciding.