---
title: "Should I rent a model API or run my own?"
description: "Renting an AI model bills per token, and running your own bills per hour, busy or not. Where the two cost the same depends on tokens a month and how busy the GPUs are."
topic: "AI systems"
chapter: "plan-and-measure"
order: 30
minutes: { plain: 6, deep: 16 }
prerequisites: ["Percentages and simple fractions", "Reading short Python functions"]
seo_title: "Should I rent an AI model API or run my own GPUs?"
date: 2026-10-20
---
# Should I rent a model API or run my own?
Renting an AI model bills per token, so a quiet month costs little. Running your own bills per hour, so the GPUs cost the same busy or idle. In this lesson's example, 300 billion tokens a month cost $39,240 to rent and $35,920 to run on 12 GPUs, because the GPUs are busy 68% of the time. Send the same tokens in 12 hours of each day instead of 24, and owning costs $57,520 against the same $39,240. This lesson builds a small calculator that finds the monthly token count where the two bills are equal, 225 billion in the first case, and shows what moves it.
## In one minute
Picture a taxi and a car you own. The taxi bills by the mile: drive little and you pay little. The car costs the same each month, parked or driving (payments, insurance, a parking space), plus someone's time to look after it and a second car for the weeks the first is in the shop. Few miles, and the taxi wins. Many miles every day, and the car's monthly cost spread over all those miles drops below the taxi's, but only if the car is really driven. A car that sits in the garage all night is still paid for.
Renting a model API is the taxi: you pay per token. Running your own is the car: you pay for each GPU for every hour of the month, plus the people who run it and a spare GPU. The cost per token of owning falls as the GPUs get busier. The break-even is the monthly token count where the two bills are equal. Below it, rent. Above it, owning can win, if the busy share holds and the people and the spare are counted.
## Explore the numbers
[Open the widget](https://spicyshark.co/learn/should-i-rent-an-api-or-run-my-own/widget.html) to set the two bills against each other. It runs the lesson's calculator in your browser.
- **The start.** It opens at 300 billion tokens a month with the traffic spread over 24 hours. Renting costs $39,240. Owning costs $35,920, with 12 GPUs busy 68% of the time. The lines cross at 225 billion tokens a month.
- **Make the load peaky.** Move the hours a day with traffic down to 4. Owning needs 62 GPUs, costs $143,920 and is busy 13% of the time. The lines never meet.
- **Change the people.** With $0 a month for people the lines cross at 49.5 billion tokens a month. With $20,000 they cross at 401 billion.
- **More settings** holds the two token prices, the price of one GPU hour and the number of spare GPUs.
The prices are examples to replace, not a price list. Go deeper says what the calculator leaves out.
## How it works
### The picture, step by step
The taxi and the car, one step at a time. The next part shows the line of code behind each step, and the part after that runs the steps on a month of real numbers.
1. **Count the month's miles.** Add up the tokens the service sends and receives in a month: these are the miles. The unit is one reply: the usual chats of [lesson 29](https://spicyshark.co/learn/how-many-gpus-for-1000-users/) average 1,882 tokens in and 157 out. At 300 billion tokens a month that is 147,130,947 replies.
2. **Rent: the taxi meter.** Each reply costs its tokens times the price per token, the sum of [lesson 7](https://spicyshark.co/learn/why-ai-bill-is-high/). At $0.10 per million tokens in and $0.50 per million out, one reply costs 266.7 millionths of a dollar.
3. **Own: cars for the busiest hour.** Spread the replies over the hours of each day that have traffic. Lesson 29's plan counts the GPUs that keep every chat under its speed target at that pace, with room in hand, and one spare GPU is added.
4. **Own: the garage bill.** A GPU is paid for every hour of the month: the GPUs times 720 hours (30 days of 24) times the price of a GPU hour, plus a stated monthly cost for the people who run them.
5. **Busy share and cost per token.** The busy share is the replies served divided by what the GPUs could serve in the month, so a parked car lowers it. The cost per million tokens of owning is its monthly cost divided by the millions of tokens, so it falls as the busy share rises.
6. **Find the break-even.** Add one more car at a time. The first count whose bill is no more than the taxi fare for the tokens it carries holds the crossing, and the code solves for it.
### Each step in code
One real line of the lesson's code does each step. The files are in `code/`, and the line numbers are those of this lesson's files.
| Step | The code line | What it does |
|---|---|---|
| 1. Count the demand | `rent_vs_own.py:68`
`return math.floor(tokens / TOKENS_PER_REPLY + 0.5)` | Divides the month's tokens by the 2,039 tokens of one reply (1,882 in, 157 out) and rounds to a whole reply. |
| 2. Rent | `rent_vs_own.py:74`
`return bill.bill(bill.Chat(system=0, message=TOKENS_IN, reply=TOKENS_OUT, turns=replies, keep=0), price_in, price_out).total` | Lesson 7's `bill` on `replies` one-turn chats with nothing carried over. Input tokens are billed at the input price and output tokens at the output price. |
| 3. Cars for the busiest hour | `rent_vs_own.py:78`
`return cp.plan(1, 100, replies / (DAYS * hours), MIX, TARGET_MS)["n"] + spare` | Hands lesson 29's `plan` the replies an hour (the month's replies over 30 days of `hours` hours) from one user who is always active. It returns the GPUs for speed and memory with room in hand, and the line adds the spare. |
| 4. The garage bill | `rent_vs_own.py:82`
`return gpus * HOURS_PER_MONTH * gpu_hour + people` | 720 hours for every GPU, whether or not a reply is being written, plus the people. |
| 5. Busy share | `rent_vs_own.py:86`
`return replies / (gpus * REPLIES_PER_GPU_SECOND * SECONDS_PER_MONTH)` | The replies served over the replies all the paid GPUs could finish in a month. `REPLIES_PER_GPU_SECOND` is one GPU's finishing rate at its limit, from lesson 29's plan. |
| 6. Break-even | `rent_vs_own.py:107`
`if own <= per_reply * n * carry:` | `own` is the bill for `n` planned GPUs and `per_reply * n * carry` is the rent for all the replies they carry. The `for` loop on line 105 tries `n` = 1, 2, 3 and so on, and the first `n` that passes holds the crossing. |
### A real run
`trace.py` runs the steps on one month. It calls the same functions as the table, and only prints what they return. Run it from the repository root:
```bash
cd lessons/should-i-rent-an-api-or-run-my-own/code && python3 trace.py
```
```text
1. demand 300,000,000,000 tokens a month = 147,130,947 replies of 1,882 in + 157 out
2. rent 147,130,947 replies at $0.10 in and $0.50 out per million tokens = $39,240 a month
3. GPUs traffic in 24 hours a day = 56.8 replies a second at the peak; lesson 29's plan: 11 GPUs + 1 spare = 12
4. own 12 GPUs x 720 hours x $3.00 = $25,920, plus $10,000 for people = $35,920 a month
5. busy 147,130,947 replies / (12 GPUs x 6.98 a second x 2,592,000 seconds) = 67.7%
per million tokens: owning $0.1197, renting $0.1308
6. cross owning first costs no more than renting at 225 billion tokens a month, with 9 GPUs paid for there
at 300 billion: owning is $3,320 cheaper than renting
the same tokens, with all the traffic in fewer hours a day
18 hours: 15 GPUs, owning $42,400 against renting $39,240, busy 54.2%, crossing 374 billion tokens
12 hours: 22 GPUs, owning $57,520 against renting $39,240, busy 36.9%, crossing none
6 hours: 42 GPUs, owning $100,720 against renting $39,240, busy 19.4%, crossing none
the same traffic, with other costs for the people (traffic all day)
$ 0 a month: crossing at 49.5 billion tokens, with 3 GPUs
$10,000 a month: crossing at 225 billion tokens, with 9 GPUs
$20,000 a month: crossing at 401 billion tokens, with 15 GPUs
```
Read it against the six steps.
- **Steps 1 and 2.** 300 billion tokens at 2,039 per reply are 147,130,947 replies. Each reply rents for 1,882 x 0.10 + 157 x 0.50 = 188.2 + 78.5 = 266.7 millionths of a dollar, which makes $39,240 for the month.
- **Step 3.** The replies spread over 30 days of 24 hours are 56.8 a second. The plan needs 11 GPUs for that, and the spare makes 12.
- **Step 4.** 12 x 720 x $3.00 is $25,920, and the $10,000 for people makes $35,920, the same whether the GPUs are busy or idle.
- **Step 5.** The 12 GPUs serve 67.7% of the replies they could finish. Per million tokens, owning costs $0.1197 and renting $0.1308, so the busy GPUs make owning cheaper per token.
- **Step 6.** Owning first costs no more than renting at 225 billion tokens a month, with 9 GPUs paid for there. At 300 billion, owning is $3,320 cheaper.
- **The fewer-hours lines.** The same tokens in 12 hours need 22 GPUs, busy 36.9% of the time, and no token count makes owning cheaper. Each $10,000 more for people, up to $20,000, pushes the crossing out by about 175 billion tokens.
The prices are examples, not a price list.
## Your turn
The trace spread the traffic over all 24 hours. Squeeze it into half the day.
1. In `code/trace.py`, change line 10 to `HOURS = 12`.
2. Run `python3 trace.py` again in the `code` folder.
Look at steps 3 to 6. How many GPUs does owning need, how busy are they, and does owning ever catch up with renting?
What you should see
The top of the output:
```text
1. demand 300,000,000,000 tokens a month = 147,130,947 replies of 1,882 in + 157 out
2. rent 147,130,947 replies at $0.10 in and $0.50 out per million tokens = $39,240 a month
3. GPUs traffic in 12 hours a day = 113.5 replies a second at the peak; lesson 29's plan: 21 GPUs + 1 spare = 22
4. own 22 GPUs x 720 hours x $3.00 = $47,520, plus $10,000 for people = $57,520 a month
5. busy 147,130,947 replies / (22 GPUs x 6.98 a second x 2,592,000 seconds) = 36.9%
per million tokens: owning $0.1917, renting $0.1308
6. cross no crossing in 12 hours a day: owning stays dearer however many tokens are sent
at 300 billion: owning is $18,280 dearer than renting
```
The same tokens now arrive twice as fast, so the plan needs 21 GPUs and a spare: 22, against 12. The bill is $57,520 against $39,240 for renting, and the GPUs are busy 36.9% of the time. There is no crossing at any volume. Each planned GPU carries at most the rent of $1,931 a month in 12-hour days, and it costs $2,160. More tokens only add GPUs that cost more than the rent they replace.
## What this means when you build
- **Count tokens from your own logs, not users.** A user count says nothing about tokens. Take a month of real requests, add up the tokens in and out, and use the bill sum of lesson 7.
- **Count the busy share honestly.** Nights and weekends are idle GPU hours you still pay for. Under the toy's prices a GPU must be busy more than about 45% of the month for owning to win, even with free people, so 12 hours a day of traffic cannot win.
- **Count the people, the spare and the on-call.** Each $10,000 a month for people moves the crossing out by about 160 to 175 billion tokens. A second spare moves it from 225 to 258 billion.
- **Rent first and measure.** A month of real logs beats a forecast. Caching ([lesson 8](https://spicyshark.co/learn/what-is-prompt-caching/)) and a smaller model can cut the rented bill, so test the cheaper model on your own questions ([lesson 13](https://spicyshark.co/learn/check-ai-answers-with-a-test-set/)) before you compare.
- **Revisit when volume or prices change.** At $4.00 for a GPU hour the crossing moves from 225 to 407 billion tokens, and at $2.00 to 143 billion.
- **Cost may not decide.** Rules on where data may be kept, privacy terms and response time can settle the choice before price does. Check them as a separate list, because this calculator weighs only cost.
## Go deeper
*Optional. The formula, the sources and the tests behind the picture, then the code in the `code/` folder. `python3 show_numbers.py` prints every toy number in this section.*
### Where the lines cross
Rent is a straight line. At $0.10 and $0.50 per million tokens it is $131 for each billion tokens. Owning is a staircase. One planned GPU carries `K` = 14,483,136 replies a month when the traffic fills 24 hours (80% of what it finishes at its limit, lesson 29's room in hand), which is 29.5 billion tokens. Past that, one more GPU is paid for, at `c` = $2,160 a month. With `F` for the fixed cost (the $10,000 for people plus the spare's $2,160) and `r` for the rent of one reply, the bill for `n` planned GPUs is `F + n x c`, and the first crossing is the first `n` with `F + n x c <= r x n x K`.
Smoothed into a line, that gives a formula for the crossing: `F / (r - c / K)` replies. Here `r` = $0.0002667 and `c / K` = $0.0001491, so the crossing is `12,160 / 0.0001176` = 103.4 million replies, or 211 billion tokens. The staircase gives 225 billion, within one GPU's carry of 29.5 billion. The formula shows two things. A bigger `F` moves the crossing out in proportion. And if `c / K` is at least `r`, a GPU costs more than the rent for the replies it carries, so the lines never cross.
The first crossing is not always the last, because a new GPU adds a step just where the lines meet. Here owning costs more than renting again between 236 and 242 billion tokens a month. The code returns the first crossing.
*Figure 1. The monthly bills over the month's tokens, traffic over 24 hours. Rent is the straight line and owning the steps.*
The search is these four lines of `break_even` in `code/rent_vs_own.py`:
```python
for n in range(1, MAX_GPUS + 1):
own = people + (n + spare) * gpu_month
if own <= per_reply * n * carry:
return {"replies": own / per_reply, "tokens": own / per_reply * TOKENS_PER_REPLY, "gpus": n + spare, "own": own}
```
### Busy share sets the cost per token
Owning costs the same however many replies the GPUs write, so its cost per million tokens is the monthly bill divided by the millions of tokens sent. Twice the replies on the same GPUs make that half as much. Rent costs $0.1308 per million tokens at every volume.
For the 12 GPUs of the first case, owning is cheaper per million tokens above 62% busy. At 68% it is $0.1197. The busy share cannot reach 100%: a planned GPU runs at 80% of its limit, the spare lowers the top for these 12 GPUs to 73%, and a GPU that gets traffic 12 hours of the day is busy at most 40%.
*Figure 2. Owning cost per million tokens against the busy share, 12 GPUs. The rent price is flat at $0.1308.*
The limit follows. One GPU costs $2,160 a month. Flat out for the month it finishes 18.1 million replies, worth $4,828 at the rent price. So even with free people, a GPU must be busy more than $2,160 / $4,828 = 44.7% of the month to beat the rent. A planned GPU is busy at most 80% x hours / 24 of it, so the hours a day with traffic must exceed 13.4. At 14 hours the crossing is 2.26 trillion tokens a month, with 132 GPUs. At 13 hours or fewer it does not exist.
### Steady against peaky
The planner sizes the GPUs for the busiest hour. Squeeze the same 300 billion tokens into fewer hours and the pace rises, so more GPUs are paid for all month, idle in the hours without traffic. The rent does not care when the tokens arrive, and is $39,240 in every row. "Cross at" is the monthly tokens, in billions, where owning first costs no more than renting; "none" means no volume does.
| Hours a day | GPUs | Own a month | Busy | Cross at |
|---|---|---|---|---|
| 24 | 12 | $35,920 | 68% | 225 |
| 18 | 15 | $42,400 | 54% | 374 |
| 16 | 17 | $46,720 | 48% | 588 |
| 12 | 22 | $57,520 | 37% | none |
| 8 | 32 | $79,120 | 25% | none |
| 4 | 62 | $143,920 | 13% | none |
*Figure 3. The cost of owning for the same tokens as the hours with traffic shrink. The dashed line is the rent.*
The toy puts all the traffic in the same hours every day, at an even pace. A real day has a peak and a quieter stretch, and a week has weekends. Take the hourly counts from your logs, and read the table as two ends of a range.
### People, the spare and the prices
The people cost, the spare and both prices move the crossing. Traffic over 24 hours, the spare at 1:
| People a month | Crossing | GPUs paid for there |
|---|---|---|
| $0 | 49.5 billion | 3 |
| $5,000 | 137 billion | 6 |
| $10,000 | 225 billion | 9 |
| $20,000 | 401 billion | 15 |
| $30,000 | 560 billion | 20 |
*Figure 4. Where owning first costs no more than renting, by the monthly cost of the people.*
Each $10,000 moves the crossing about 160 to 175 billion tokens. The spare matters less: none gives 176 billion, one 225 and two 258. A GPU hour at $2.00 gives 143 billion tokens and at $4.00 gives 407. Rent at $2 and $10 per million tokens gives 5.47 billion, so at 300 billion tokens renting costs $784,796 and owning $35,920. Rent at half the toy's prices, $0.05 and $0.25, leaves no crossing: at 300 billion tokens renting costs $19,620 and owning $35,920.
### Where the toy's prices sit
The toy's prices are round numbers. They are not quotes, and providers change them. Two published price pages, read on 2026-10-20, show where the toy's prices sit.
- Anthropic's pricing page lists base prices per million tokens from $0.10 to $15 for input and from $0.50 to $75 for output across its listed model rows, retired rows included. The toy's $0.10 and $0.50 equal the cheapest row, which applies to prompts up to 100,000 tokens, and the longest prompt in lesson 29's mix is about 6,000 tokens. Most rows cost far more, and the sums above show the effect: at $2 and $10 per million tokens the crossing falls to 5.47 billion tokens a month. The page also lists a batch discount of 50% on both input and output tokens, and a lower price for cached input.
- Lambda's pricing page lists GPU instances from $0.69 to $6.99 per GPU per hour, plus sales tax, VAT or GST where it applies. The toy's $3.00 sits inside that range. The page lists the 80 GB A100 in its 8-GPU instance at $2.79.
### What the toy leaves out
- **Buying.** The toy rents its GPUs by the hour. Buying adds a purchase price and depreciation, and power, space and cooling.
- **Stopping GPUs off-peak.** The toy pays for every GPU all month. A GPU that could be stopped at night would cost less, which would move the peaky rows toward the rent.
- **Smaller models on cheaper GPUs.** A smaller or [quantized](https://spicyshark.co/learn/quantization/) model can run on less hardware, and lowers `c / K`.
- **Cheaper rent.** The batch discount and the cached-input price on the page above lower the rented bill and move the crossing out. For the model priced at $0.10 and $0.50, the page lists a batch price of $0.05 and $0.25 for prompts up to 100,000 tokens, and at those prices there is no crossing.
- **Equal answers.** The sums compare equal volume. Whether two models give equally good answers is a test-set question ([lesson 13](https://spicyshark.co/learn/check-ai-answers-with-a-test-set/)).
- **Equal text, unequal tokens.** The Anthropic page says its newer models use a tokenizer that produces approximately 30% more tokens for the same text. A token-for-token comparison across models is approximate.
- **One model, one GPU kind, one chat mix and an even pace.** Real traffic has bursts, failures and model upgrades, and the people cost is a single number.
### Proof it works
```bash
cd lessons/should-i-rent-an-api-or-run-my-own/code && python3 -m pytest -q
```
The tests check the tokens and the rent against sums made by hand, the GPU count against lesson 29's plan, the owning bill and the busy share by identity, and the break-even as the first token count where owning is no dearer, by searching below it. They check the effect of the people cost against the formula above. They also check that every number in this lesson (the prose, the trace, the figures and their words) appears in the code's output, that each `file:line` in the code table holds the line it quotes, that `trace.py` prints the block above and that the one-line change in Your turn gives the answer shown, and that the figures on disk match `make_figs.py`. One test runs the widget's JavaScript under Node.js (so it needs Node.js installed) on a grid of settings and requires the same words and chart as the Python.
## Skills this lesson builds
- Turn a month of traffic into tokens, a rented bill and an owned bill, and find the token count where they cross.
- Read a busy share and a cost per million tokens, and say what each hides.
- Say which costs the comparison leaves out: people, spare capacity, purchase, power, and the quality of the answers.
- Show how the answer moves with hours of traffic, people and prices.
**Portfolio project:** write a break-even calculator that reads a month of request logs (hour, tokens in, tokens out), counts the GPUs from a measured speed table, and prints the rented bill, the owned bill, the busy share and the crossing for your own price list. Publish the code and one chart of the two bills over monthly tokens, with a note on what the calculator leaves out.
## Check yourself
1. At $0.10 per million tokens in and $0.50 per million out, one reply of 1,882 tokens in and 157 out costs how much to rent? What do a million of them cost?
Answer
1,882 x 0.10 = 188.2 and 157 x 0.50 = 78.5 millionths of a dollar, which is 266.7 millionths per reply. A million replies cost $266.70.Answer
$2,160 / 0.0002667 = 8.1 million replies. 8.1 / 18.1 = 44.7% of the month. Below that the GPU costs more than the rent for its replies, even with free people.Answer
The pace doubles to 113.5 replies a second, so the plan needs 21 GPUs and a spare: 22, not 12. 22 x 720 x $3.00 = $47,520, plus $10,000 for people. The GPUs are idle half of each day and still paid for, and the rent stays $39,240.Answer
From 225 to 401 billion tokens a month, with 15 GPUs paid for there. The fixed cost is paid before the first reply, and each token saves only a little against the rent, so a bigger fixed cost takes more tokens to pay back: about 175 billion tokens for each $10,000.Answer
20 x 2,592,000 seconds = 51,840,000 replies, or 105.7 billion tokens, below the 225 billion crossing. Renting costs $13,826. Owning takes 5 GPUs: 5 x 720 x $3.00 = $10,800, plus $10,000 for people = $20,800. Renting is cheaper, and a real day with a quieter night would only favor renting more.