What a million tokens costs, both ways.

The short version

Almost every comparison of these two you will read comes out as a single number — "self-hosting is ten times cheaper", "the API is cheaper until you are enormous" — and both are the same mistake, which is quoting a cost per token for a machine that is not billed per token. A rented GPU has exactly one price and it does not move: whether you generate a billion tokens this month or none, the invoice is identical. So the honest question is not "what does a token cost here" but "how many tokens do I need to generate before the fixed price beats the meter".

That number is a division, and it is done below on our own prices and our own throughput model — the same one the configurator uses. Everything after it is about the two things that decide whether you can reach that volume: batching, and the hours you leave the machine idle.

The arithmetic

Four lines. The third is the one worth writing on a whiteboard; the fourth is the one people skip, and skipping it is what turns a sound estimate into a wrong one.

# 1. What the API charges. Linear in what you use, zero when you stop.
api_cost_per_month = output_tokens_per_month / 1e6 × api_price_per_million

# 2. What the machine charges. A constant. Tokens are free once you own the hours.
machine_cost_per_month = monthly_price

# 3. Set them equal. This is the break-even VOLUME, in output tokens per month.
break_even_tokens = monthly_price / api_price_per_million × 1e6

# 4. And the only question that matters: can the machine produce that many?
#    730 hours is the month we bill, so 2,628,000 seconds of it.
sustained_tokens_per_second = break_even_tokens / 2,628,000

Line 4 is a sustained rate, not a peak. A machine that hits 600 tokens a second for one hour a day and idles for the other twenty-three has a sustained rate of 25, and the invoice does not care which of the two you talk about.

What one stream costs

Start with the worst case, because it is the only figure that can be stated without assumptions: one request at a time, no batching, the machine otherwise idle. Every node below runs Llama 3.3 70B at 4-bit (AWQ, GPTQ), and the throughput is our own conservative estimate — the one that is bounded by memory bandwidth rather than measured on a good day.

Read this as a ceiling, never as a quote. It is what a token costs if you make the machine answer one question at a time and leave it alone the rest of the month — the least efficient way to own a GPU that exists. Every real serving stack does better, and the next two sections are about how much better.

The break-even, per tier

Take the cheapest row above — 2 × NVIDIA RTX 5090 at $692 a month — and ask how much work it has to do to beat each tier of the API market.

The last column is the whole article. Where it says "already cheaper", one un-batched stream on a $692 machine beats the API and there is nothing to think about. Where it shows a multiple, the machine can still win — but only if you actually feed it, which is the next section.

Batching is the whole game

Generating one token requires reading the active weights out of memory once. Generating a hundred tokens for a hundred different requests requires reading them once as well — the same weights serve every request in the batch. That is why a serving stack with continuous batching produces many times the tokens per second of a single stream on identical hardware, and why the ratio in the last column of that table is reachable at all.

The practical consequence: treat the multiple in the break-even table as a throughput target, not a user count. A target of ten times single-stream does not mean "ten users"; it means the server has to average ten times its un-batched rate, which usually takes rather more than ten concurrent requests and rather fewer than a hundred. Measure it on the machine before you commit to a number — it is one benchmark, and it settles the argument.

The idle hours you still pay for

Everything above assumed the load is spread evenly across the month. It never is. A machine rented by the month is paid for on Sunday at 04:00, and the tokens you did not generate then do not carry forward. Divide the week you actually serve into the week you actually pay for:

The third row is where most internal tools live, and it quietly multiplies your break-even by more than four. That is the single most common reason a self-hosting estimate turns out wrong — not the token price, not the hardware, but the assumption that a working week is a whole week.

Why your prompt is nearly free

One asymmetry works strongly in favour of owning the machine, and no price-per-token comparison shows it. An API charges you for input tokens — usually a quarter to a third of the output price, per token. On your own GPU, input tokens are processed in the prefill pass, which handles the entire prompt in parallel and is limited by arithmetic rather than by memory bandwidth. Prefill runs at an order of magnitude more tokens per second than generation does.

What the arithmetic leaves out

Four things that never appear in a cost-per-token table, and that decide the question at least as often as the numbers do.

Which side you are on

In order, and stop at the first one that describes you.

1. You need a frontier closed model's quality. Then this article does not apply: nobody rents you those weights. Use the API, and revisit when an open model closes the gap on your specific task.
2. Your volume is low or you do not know it yet. Use the API. It is the cheapest way to find out what your traffic actually looks like, and the meter is a perfectly good measuring instrument.
3. Your traffic is spiky and tolerates a queue. Use the API, or split it: your own machine for the steady base load, a hosted endpoint for the peaks. Nothing forces the decision to be all one way.
4. You are sustaining real, batched volume against an open-weight model. This is the case the hardware wins, and it wins by a lot — the fixed price stops moving while the meter keeps running. Check your own number in the table above before you believe it.
5. The prompts are sensitive, the model must not change, or the bill must be knowable in advance. Then the arithmetic is a formality. Rent the machine that holds your model, and pay for it without telling anyone who you are.

Whichever line you stopped on, the number to check is the one in the break-even table for your model and your volume. The configurator reports, for every node we rent, whether your model fits on a single card, has to be split, and what the throughput estimate is — which is every input this arithmetic needs. If you are still deciding between renting by the month and metering by the hour, that is a different break-even, and it is worked out too.