On this page
Renting an AI model bills per token, so a quiet month costs little. Running your own bills per hour, so the GPUs cost the same busy or idle. In this lesson’s example, 300 billion tokens a month cost $39,240 to rent and $35,920 to run on 12 GPUs, because the GPUs are busy 68% of the time. Send the same tokens in 12 hours of each day instead of 24, and owning costs $57,520 against the same $39,240. This lesson builds a small calculator that finds the monthly token count where the two bills are equal, 225 billion in the first case, and shows what moves it.
In one minute
Picture a taxi and a car you own. The taxi bills by the mile: drive little and you pay little. The car costs the same each month, parked or driving (payments, insurance, a parking space), plus someone’s time to look after it and a second car for the weeks the first is in the shop. Few miles, and the taxi wins. Many miles every day, and the car’s monthly cost spread over all those miles drops below the taxi’s, but only if the car is really driven. A car that sits in the garage all night is still paid for.
Renting a model API is the taxi: you pay per token. Running your own is the car: you pay for each GPU for every hour of the month, plus the people who run it and a spare GPU. The cost per token of owning falls as the GPUs get busier. The break-even is the monthly token count where the two bills are equal. Below it, rent. Above it, owning can win, if the busy share holds and the people and the spare are counted.
Explore the numbers
Open the widget to set the two bills against each other. It runs the lesson’s calculator in your browser.
- The start. It opens at 300 billion tokens a month with the traffic spread over 24 hours. Renting costs $39,240. Owning costs $35,920, with 12 GPUs busy 68% of the time. The lines cross at 225 billion tokens a month.
- Make the load peaky. Move the hours a day with traffic down to 4. Owning needs 62 GPUs, costs $143,920 and is busy 13% of the time. The lines never meet.
- Change the people. With $0 a month for people the lines cross at 49.5 billion tokens a month. With $20,000 they cross at 401 billion.
- More settings holds the two token prices, the price of one GPU hour and the number of spare GPUs.
The prices are examples to replace, not a price list. Go deeper says what the calculator leaves out.
How it works
The picture, step by step
The taxi and the car, one step at a time. The next part shows the line of code behind each step, and the part after that runs the steps on a month of real numbers.
- Count the month’s miles. Add up the tokens the service sends and receives in a month: these are the miles. The unit is one reply: the usual chats of lesson 29 average 1,882 tokens in and 157 out. At 300 billion tokens a month that is 147,130,947 replies.
- Rent: the taxi meter. Each reply costs its tokens times the price per token, the sum of lesson 7. At $0.10 per million tokens in and $0.50 per million out, one reply costs 266.7 millionths of a dollar.
- Own: cars for the busiest hour. Spread the replies over the hours of each day that have traffic. Lesson 29’s plan counts the GPUs that keep every chat under its speed target at that pace, with room in hand, and one spare GPU is added.
- Own: the garage bill. A GPU is paid for every hour of the month: the GPUs times 720 hours (30 days of 24) times the price of a GPU hour, plus a stated monthly cost for the people who run them.
- Busy share and cost per token. The busy share is the replies served divided by what the GPUs could serve in the month, so a parked car lowers it. The cost per million tokens of owning is its monthly cost divided by the millions of tokens, so it falls as the busy share rises.
- Find the break-even. Add one more car at a time. The first count whose bill is no more than the taxi fare for the tokens it carries holds the crossing, and the code solves for it.
Each step in code
One real line of the lesson’s code does each step. The files are in code/, and the line numbers are those of this lesson’s files.
| Step | The code line | What it does |
|---|---|---|
| 1. Count the demand | rent_vs_own.py:68return math.floor(tokens / TOKENS_PER_REPLY + 0.5) |
Divides the month’s tokens by the 2,039 tokens of one reply (1,882 in, 157 out) and rounds to a whole reply. |
| 2. Rent | rent_vs_own.py:74return bill.bill(bill.Chat(system=0, message=TOKENS_IN, reply=TOKENS_OUT, turns=replies, keep=0), price_in, price_out).total |
Lesson 7’s bill on replies one-turn chats with nothing carried over. Input tokens are billed at the input price and output tokens at the output price. |
| 3. Cars for the busiest hour | rent_vs_own.py:78return cp.plan(1, 100, replies / (DAYS * hours), MIX, TARGET_MS)["n"] + spare |
Hands lesson 29’s plan the replies an hour (the month’s replies over 30 days of hours hours) from one user who is always active. It returns the GPUs for speed and memory with room in hand, and the line adds the spare. |
| 4. The garage bill | rent_vs_own.py:82return gpus * HOURS_PER_MONTH * gpu_hour + people |
720 hours for every GPU, whether or not a reply is being written, plus the people. |
| 5. Busy share | rent_vs_own.py:86return replies / (gpus * REPLIES_PER_GPU_SECOND * SECONDS_PER_MONTH) |
The replies served over the replies all the paid GPUs could finish in a month. REPLIES_PER_GPU_SECOND is one GPU’s finishing rate at its limit, from lesson 29’s plan. |
| 6. Break-even | rent_vs_own.py:107if own <= per_reply * n * carry: |
own is the bill for n planned GPUs and per_reply * n * carry is the rent for all the replies they carry. The for loop on line 105 tries n = 1, 2, 3 and so on, and the first n that passes holds the crossing. |
A real run
trace.py runs the steps on one month. It calls the same functions as the table, and only prints what they return. Run it from the repository root:
cd lessons/should-i-rent-an-api-or-run-my-own/code && python3 trace.py1. demand 300,000,000,000 tokens a month = 147,130,947 replies of 1,882 in + 157 out
2. rent 147,130,947 replies at $0.10 in and $0.50 out per million tokens = $39,240 a month
3. GPUs traffic in 24 hours a day = 56.8 replies a second at the peak; lesson 29's plan: 11 GPUs + 1 spare = 12
4. own 12 GPUs x 720 hours x $3.00 = $25,920, plus $10,000 for people = $35,920 a month
5. busy 147,130,947 replies / (12 GPUs x 6.98 a second x 2,592,000 seconds) = 67.7%
per million tokens: owning $0.1197, renting $0.1308
6. cross owning first costs no more than renting at 225 billion tokens a month, with 9 GPUs paid for there
at 300 billion: owning is $3,320 cheaper than renting
the same tokens, with all the traffic in fewer hours a day
18 hours: 15 GPUs, owning $42,400 against renting $39,240, busy 54.2%, crossing 374 billion tokens
12 hours: 22 GPUs, owning $57,520 against renting $39,240, busy 36.9%, crossing none
6 hours: 42 GPUs, owning $100,720 against renting $39,240, busy 19.4%, crossing none
the same traffic, with other costs for the people (traffic all day)
$ 0 a month: crossing at 49.5 billion tokens, with 3 GPUs
$10,000 a month: crossing at 225 billion tokens, with 9 GPUs
$20,000 a month: crossing at 401 billion tokens, with 15 GPUsRead it against the six steps.
- Steps 1 and 2. 300 billion tokens at 2,039 per reply are 147,130,947 replies. Each reply rents for 1,882 x 0.10 + 157 x 0.50 = 188.2 + 78.5 = 266.7 millionths of a dollar, which makes $39,240 for the month.
- Step 3. The replies spread over 30 days of 24 hours are 56.8 a second. The plan needs 11 GPUs for that, and the spare makes 12.
- Step 4. 12 x 720 x $3.00 is $25,920, and the $10,000 for people makes $35,920, the same whether the GPUs are busy or idle.
- Step 5. The 12 GPUs serve 67.7% of the replies they could finish. Per million tokens, owning costs $0.1197 and renting $0.1308, so the busy GPUs make owning cheaper per token.
- Step 6. Owning first costs no more than renting at 225 billion tokens a month, with 9 GPUs paid for there. At 300 billion, owning is $3,320 cheaper.
- The fewer-hours lines. The same tokens in 12 hours need 22 GPUs, busy 36.9% of the time, and no token count makes owning cheaper. Each $10,000 more for people, up to $20,000, pushes the crossing out by about 175 billion tokens.
The prices are examples, not a price list.
Hands on
Your turn
The trace spread the traffic over all 24 hours. Squeeze it into half the day.
- In
code/trace.py, change line 10 toHOURS = 12. - Run
python3 trace.pyagain in thecodefolder.
Look at steps 3 to 6. How many GPUs does owning need, how busy are they, and does owning ever catch up with renting?
What you should see
The top of the output:
1. demand 300,000,000,000 tokens a month = 147,130,947 replies of 1,882 in + 157 out
2. rent 147,130,947 replies at $0.10 in and $0.50 out per million tokens = $39,240 a month
3. GPUs traffic in 12 hours a day = 113.5 replies a second at the peak; lesson 29's plan: 21 GPUs + 1 spare = 22
4. own 22 GPUs x 720 hours x $3.00 = $47,520, plus $10,000 for people = $57,520 a month
5. busy 147,130,947 replies / (22 GPUs x 6.98 a second x 2,592,000 seconds) = 36.9%
per million tokens: owning $0.1917, renting $0.1308
6. cross no crossing in 12 hours a day: owning stays dearer however many tokens are sent
at 300 billion: owning is $18,280 dearer than rentingThe same tokens now arrive twice as fast, so the plan needs 21 GPUs and a spare: 22, against 12. The bill is $57,520 against $39,240 for renting, and the GPUs are busy 36.9% of the time. There is no crossing at any volume. Each planned GPU carries at most the rent of $1,931 a month in 12-hour days, and it costs $2,160. More tokens only add GPUs that cost more than the rent they replace.
What this means when you build
- Count tokens from your own logs, not users. A user count says nothing about tokens. Take a month of real requests, add up the tokens in and out, and use the bill sum of lesson 7.
- Count the busy share honestly. Nights and weekends are idle GPU hours you still pay for. Under the toy’s prices a GPU must be busy more than about 45% of the month for owning to win, even with free people, so 12 hours a day of traffic cannot win.
- Count the people, the spare and the on-call. Each $10,000 a month for people moves the crossing out by about 160 to 175 billion tokens. A second spare moves it from 225 to 258 billion.
- Rent first and measure. A month of real logs beats a forecast. Caching (lesson 8) and a smaller model can cut the rented bill, so test the cheaper model on your own questions (lesson 13) before you compare.
- Revisit when volume or prices change. At $4.00 for a GPU hour the crossing moves from 225 to 407 billion tokens, and at $2.00 to 143 billion.
- Cost may not decide. Rules on where data may be kept, privacy terms and response time can settle the choice before price does. Check them as a separate list, because this calculator weighs only cost.
Optionalabout 16 min
Go deeper
Optional. The formula, the sources and the tests behind the picture, then the code in the code/ folder. python3 show_numbers.py prints every toy number in this section.
Where the lines cross
Rent is a straight line. At $0.10 and $0.50 per million tokens it is $131 for each billion tokens. Owning is a staircase. One planned GPU carries K = 14,483,136 replies a month when the traffic fills 24 hours (80% of what it finishes at its limit, lesson 29’s room in hand), which is 29.5 billion tokens. Past that, one more GPU is paid for, at c = $2,160 a month. With F for the fixed cost (the $10,000 for people plus the spare’s $2,160) and r for the rent of one reply, the bill for n planned GPUs is F + n x c, and the first crossing is the first n with F + n x c <= r x n x K.
Smoothed into a line, that gives a formula for the crossing: F / (r - c / K) replies. Here r = $0.0002667 and c / K = $0.0001491, so the crossing is 12,160 / 0.0001176 = 103.4 million replies, or 211 billion tokens. The staircase gives 225 billion, within one GPU’s carry of 29.5 billion. The formula shows two things. A bigger F moves the crossing out in proportion. And if c / K is at least r, a GPU costs more than the rent for the replies it carries, so the lines never cross.
The first crossing is not always the last, because a new GPU adds a step just where the lines meet. Here owning costs more than renting again between 236 and 242 billion tokens a month. The code returns the first crossing.
Figure 1. The monthly bills over the month’s tokens, traffic over 24 hours. Rent is the straight line and owning the steps.
The search is these four lines of break_even in code/rent_vs_own.py:
for n in range(1, MAX_GPUS + 1):
own = people + (n + spare) * gpu_month
if own <= per_reply * n * carry:
return {"replies": own / per_reply, "tokens": own / per_reply * TOKENS_PER_REPLY, "gpus": n + spare, "own": own}Busy share sets the cost per token
Owning costs the same however many replies the GPUs write, so its cost per million tokens is the monthly bill divided by the millions of tokens sent. Twice the replies on the same GPUs make that half as much. Rent costs $0.1308 per million tokens at every volume.
For the 12 GPUs of the first case, owning is cheaper per million tokens above 62% busy. At 68% it is $0.1197. The busy share cannot reach 100%: a planned GPU runs at 80% of its limit, the spare lowers the top for these 12 GPUs to 73%, and a GPU that gets traffic 12 hours of the day is busy at most 40%.
Figure 2. Owning cost per million tokens against the busy share, 12 GPUs. The rent price is flat at $0.1308.
The limit follows. One GPU costs $2,160 a month. Flat out for the month it finishes 18.1 million replies, worth $4,828 at the rent price. So even with free people, a GPU must be busy more than $2,160 / $4,828 = 44.7% of the month to beat the rent. A planned GPU is busy at most 80% x hours / 24 of it, so the hours a day with traffic must exceed 13.4. At 14 hours the crossing is 2.26 trillion tokens a month, with 132 GPUs. At 13 hours or fewer it does not exist.
Steady against peaky
The planner sizes the GPUs for the busiest hour. Squeeze the same 300 billion tokens into fewer hours and the pace rises, so more GPUs are paid for all month, idle in the hours without traffic. The rent does not care when the tokens arrive, and is $39,240 in every row. “Cross at” is the monthly tokens, in billions, where owning first costs no more than renting; “none” means no volume does.
| Hours a day | GPUs | Own a month | Busy | Cross at |
|---|---|---|---|---|
| 24 | 12 | $35,920 | 68% | 225 |
| 18 | 15 | $42,400 | 54% | 374 |
| 16 | 17 | $46,720 | 48% | 588 |
| 12 | 22 | $57,520 | 37% | none |
| 8 | 32 | $79,120 | 25% | none |
| 4 | 62 | $143,920 | 13% | none |
Figure 3. The cost of owning for the same tokens as the hours with traffic shrink. The dashed line is the rent.
The toy puts all the traffic in the same hours every day, at an even pace. A real day has a peak and a quieter stretch, and a week has weekends. Take the hourly counts from your logs, and read the table as two ends of a range.
People, the spare and the prices
The people cost, the spare and both prices move the crossing. Traffic over 24 hours, the spare at 1:
| People a month | Crossing | GPUs paid for there |
|---|---|---|
| $0 | 49.5 billion | 3 |
| $5,000 | 137 billion | 6 |
| $10,000 | 225 billion | 9 |
| $20,000 | 401 billion | 15 |
| $30,000 | 560 billion | 20 |
Figure 4. Where owning first costs no more than renting, by the monthly cost of the people.
Each $10,000 moves the crossing about 160 to 175 billion tokens. The spare matters less: none gives 176 billion, one 225 and two 258. A GPU hour at $2.00 gives 143 billion tokens and at $4.00 gives 407. Rent at $2 and $10 per million tokens gives 5.47 billion, so at 300 billion tokens renting costs $784,796 and owning $35,920. Rent at half the toy’s prices, $0.05 and $0.25, leaves no crossing: at 300 billion tokens renting costs $19,620 and owning $35,920.
Where the toy’s prices sit
The toy’s prices are round numbers. They are not quotes, and providers change them. Two published price pages, read on 2026-10-20, show where the toy’s prices sit.
- Anthropic’s pricing page lists base prices per million tokens from $0.10 to $15 for input and from $0.50 to $75 for output across its listed model rows, retired rows included. The toy’s $0.10 and $0.50 equal the cheapest row, which applies to prompts up to 100,000 tokens, and the longest prompt in lesson 29’s mix is about 6,000 tokens. Most rows cost far more, and the sums above show the effect: at $2 and $10 per million tokens the crossing falls to 5.47 billion tokens a month. The page also lists a batch discount of 50% on both input and output tokens, and a lower price for cached input.
- Lambda’s pricing page lists GPU instances from $0.69 to $6.99 per GPU per hour, plus sales tax, VAT or GST where it applies. The toy’s $3.00 sits inside that range. The page lists the 80 GB A100 in its 8-GPU instance at $2.79.
What the toy leaves out
- Buying. The toy rents its GPUs by the hour. Buying adds a purchase price and depreciation, and power, space and cooling.
- Stopping GPUs off-peak. The toy pays for every GPU all month. A GPU that could be stopped at night would cost less, which would move the peaky rows toward the rent.
- Smaller models on cheaper GPUs. A smaller or quantized model can run on less hardware, and lowers
c / K. - Cheaper rent. The batch discount and the cached-input price on the page above lower the rented bill and move the crossing out. For the model priced at $0.10 and $0.50, the page lists a batch price of $0.05 and $0.25 for prompts up to 100,000 tokens, and at those prices there is no crossing.
- Equal answers. The sums compare equal volume. Whether two models give equally good answers is a test-set question (lesson 13).
- Equal text, unequal tokens. The Anthropic page says its newer models use a tokenizer that produces approximately 30% more tokens for the same text. A token-for-token comparison across models is approximate.
- One model, one GPU kind, one chat mix and an even pace. Real traffic has bursts, failures and model upgrades, and the people cost is a single number.
Proof it works
cd lessons/should-i-rent-an-api-or-run-my-own/code && python3 -m pytest -qThe tests check the tokens and the rent against sums made by hand, the GPU count against lesson 29’s plan, the owning bill and the busy share by identity, and the break-even as the first token count where owning is no dearer, by searching below it. They check the effect of the people cost against the formula above. They also check that every number in this lesson (the prose, the trace, the figures and their words) appears in the code’s output, that each file:line in the code table holds the line it quotes, that trace.py prints the block above and that the one-line change in Your turn gives the answer shown, and that the figures on disk match make_figs.py. One test runs the widget’s JavaScript under Node.js (so it needs Node.js installed) on a grid of settings and requires the same words and chart as the Python.
Skills this lesson builds
- Turn a month of traffic into tokens, a rented bill and an owned bill, and find the token count where they cross.
- Read a busy share and a cost per million tokens, and say what each hides.
- Say which costs the comparison leaves out: people, spare capacity, purchase, power, and the quality of the answers.
- Show how the answer moves with hours of traffic, people and prices.
Portfolio project: write a break-even calculator that reads a month of request logs (hour, tokens in, tokens out), counts the GPUs from a measured speed table, and prints the rented bill, the owned bill, the busy share and the crossing for your own price list. Publish the code and one chart of the two bills over monthly tokens, with a note on what the calculator leaves out.
Check yourself
-
At $0.10 per million tokens in and $0.50 per million out, one reply of 1,882 tokens in and 157 out costs how much to rent? What do a million of them cost?
Answer
1,882 x 0.10 = 188.2 and 157 x 0.50 = 78.5 millionths of a dollar, which is 266.7 millionths per reply. A million replies cost $266.70.
-
One GPU costs $3.00 an hour, so $2,160 for a 720-hour month. At 266.7 millionths of a dollar per reply, how many replies must it serve to be worth its price? It finishes 6.98 replies a second at its limit (18.1 million in a month). What busy share is that?
Answer
$2,160 / 0.0002667 = 8.1 million replies. 8.1 / 18.1 = 44.7% of the month. Below that the GPU costs more than the rent for its replies, even with free people.
-
The same 300 billion tokens arrive in 12 hours a day instead of 24. Why does owning cost $57,520 and not $35,920?
Answer
The pace doubles to 113.5 replies a second, so the plan needs 21 GPUs and a spare: 22, not 12. 22 x 720 x $3.00 = $47,520, plus $10,000 for people. The GPUs are idle half of each day and still paid for, and the rent stays $39,240.
-
The people cost $20,000 a month instead of $10,000. Where does the crossing go, and why?
Answer
From 225 to 401 billion tokens a month, with 15 GPUs paid for there. The fixed cost is paid before the first reply, and each token saves only a little against the rent, so a bigger fixed cost takes more tokens to pay back: about 175 billion tokens for each $10,000.
-
Lesson 29’s 1,000 users make 20 replies a second at the busiest hour. Suppose that pace held all day. How many tokens a month is that, and is renting or owning cheaper?
Answer
20 x 2,592,000 seconds = 51,840,000 replies, or 105.7 billion tokens, below the 225 billion crossing. Renting costs $13,826. Owning takes 5 GPUs: 5 x 720 x $3.00 = $10,800, plus $10,000 for people = $20,800. Renting is cheaper, and a real day with a quieter night would only favor renting more.
Further reading
Web pages were read on 2026-10-20; providers change them.
Gentle
- Why is my AI app bill higher than I expected? (lesson 7, the per-token bill)
- How many GPUs do I need for 1,000 users? (lesson 29, the plan this lesson prices)
- What is prompt caching, and when does it save money? (lesson 8, a way to lower the rented bill)
Deep
Optional
Code
Each file runs on its own. Download it, or read it here.
Show code
"""Keep the numbers and settings inside ../widget.html equal to the ones in rent_vs_own.py.
python3 embed_widget.py rewrite the data block in widget.html
python3 embed_widget.py --check exit 1 if widget.html differs (test_widget.py runs the same check)
"""
import json
import re
import sys
from pathlib import Path
import rent_vs_own as rvo
WIDGET = Path(__file__).resolve().parent.parent / "widget.html"
CONFIG_BLOCK = re.compile(r'(<script type="application/json" id="cp-config">)(.*?)(</script>)', re.S)
def config():
"""What the page needs to repeat the sums: lesson 29's three planner numbers, the sizes of a reply, the month, the chart's box, the values each control offers and the
opening settings."""
return {
"stay": rvo.DESIGN["stay"], "speed": rvo.DESIGN["speed"], "memory": rvo.DESIGN["memory"], "headroom": rvo.HEADROOM, "rate": rvo.REPLIES_PER_GPU_SECOND,
"tokensIn": rvo.TOKENS_IN, "tokensOut": rvo.TOKENS_OUT, "tokensPerReply": rvo.TOKENS_PER_REPLY, "days": rvo.DAYS, "hoursPerMonth": rvo.HOURS_PER_MONTH,
"maxGpus": rvo.MAX_GPUS, "box": [rvo.W, rvo.H, rvo.PAD], "maxSteps": rvo.MAX_STEPS,
"tokensB": rvo.TOKENS_B, "hours": list(rvo.HOURS_RANGE), "people": list(rvo.PEOPLE_RANGE), "pricesIn": rvo.PRICES_IN, "pricesOut": rvo.PRICES_OUT,
"gpuHours": rvo.GPU_HOURS, "spares": rvo.SPARES, "opening": rvo.OPENING,
}
def embed(html):
"""html with the data block rewritten from rent_vs_own.py."""
settings = json.dumps(config(), ensure_ascii=False, separators=(", ", ": "))
return CONFIG_BLOCK.sub(lambda m: m.group(1) + "\n" + settings + "\n" + m.group(3), html, count=1)
if __name__ == "__main__":
current = WIDGET.read_text(encoding="utf-8")
updated = embed(current)
if "--check" in sys.argv:
sys.exit(0 if updated == current else 1)
WIDGET.write_text(updated, encoding="utf-8")
print("widget.html data block now matches rent_vs_own.py")Show code
"""Draw the four figures of lesson 30 from rent_vs_own.py, so that no number in a figure is typed by hand.
python3 make_figs.py rewrite ../figs/*.svg
python3 make_figs.py --check exit 1 if a figure on disk differs (test_rent_vs_own.py runs the same check)
All four figures are 540 units wide. Text is at least 22 units (20 for axis ticks) in ink or grey on white or a light fill; orange and green are used on marks
only. Rent and owning differ by colour and by line style (a plain line against steps), and the bars say "cheaper" or "dearer" in words.
"""
import math
import sys
from pathlib import Path
from xml.sax.saxutils import escape
import rent_vs_own as rvo
FIGS = Path(__file__).resolve().parent.parent / "figs"
INK, GREY, ORANGE, GREEN, FILL, PANEL = "#0a1626", "#44566b", "#b36200", "#00855F", "#d2dae4", "#ebf0f6"
DASH = "10 8"
WIDTH = 540
TOKENS = 300 * 10**9 # the case the trace and the widget open on
def num(x: float) -> str:
return f"{x:.2f}".rstrip("0").rstrip(".")
# The SVG namespace is a name that every standalone .svg file must carry, not an address this program visits. It is written in two pieces so that a plain
# search of code/ for the start of a web address finds nothing (test_rent_vs_own.py runs that search).
NAMESPACE = "ht" + "tp://www.w3.org/2000/svg"
def frame(width: int, height: int, title: str, desc: str, body: list[str]) -> str:
head = (f'<svg xmlns="{NAMESPACE}" viewBox="0 0 {width} {height}" width="{width}" height="{height}" role="img" '
f'font-family="system-ui, sans-serif" font-size="22" fill="{GREY}">')
return "\n".join([head, f" <title>{escape(title)}</title>", f" <desc>{escape(desc)}</desc>", ' <rect width="100%" height="100%" fill="#ffffff"/>',
*(" " + line for line in body), "</svg>", ""])
def text(x: float, y: float, content: str, anchor: str = "start", color: str = INK, bold: bool = False, size: int = 22, halo: bool = False) -> str:
extra = (f' text-anchor="{anchor}"' if anchor != "start" else "") + (' font-weight="bold"' if bold else "") + (f' font-size="{size}"' if size != 22 else "")
extra += ' stroke="#ffffff" stroke-width="6" stroke-linejoin="round" paint-order="stroke"' if halo else "" # white outline, so a line behind the text does not cut through it
return f'<text x="{num(x)}" y="{num(y)}"{extra} fill="{color}">{escape(content)}</text>'
def money(x) -> str:
return "$" + rvo.commas(rvo.dollars(x))
def billions(tokens: float) -> str:
return f"{math.floor(tokens / 1e9 + 0.5):,}"
# --- figure 1: both bills over the month's tokens --------------------------------------------------------------------------------------------
def fig_lines() -> str:
per_reply, gpu_month, carry = rvo.rates(24)
step = carry * rvo.TOKENS_PER_REPLY # tokens one planned GPU carries in a month
cross = rvo.break_even(24)
now = rvo.case(TOKENS, 24)
span = 1.25 * TOKENS
steps = math.ceil(span / step)
own_at = lambda n: rvo.PEOPLE + (n + rvo.SPARE) * gpu_month
rent_at = lambda t: per_reply * t / rvo.TOKENS_PER_REPLY
ymax = math.ceil(max(rent_at(span), own_at(steps)) / 10_000) * 10_000
x0, x1, top, base = 96, 520, 60, 300
px = lambda t: x0 + t / span * (x1 - x0)
py = lambda cost: base - cost / ymax * (base - top)
body = [text(20, 36, "dollars a month", color=GREY)]
for tick in range(0, ymax + 1, 20_000):
body.append(f'<path d="M{x0} {num(py(tick))}H{x1}" stroke="{FILL}" stroke-width="2"/>')
body.append(text(x0 - 8, py(tick) + 7, rvo.commas(tick), "end", GREY, size=20))
stairs = []
for k in range(steps):
stairs += [(px(k * step), py(own_at(k + 1))), (px(min((k + 1) * step, span)), py(own_at(k + 1)))]
body.append(f'<path d="M{" L".join(f"{num(a)} {num(b)}" for a, b in stairs)}" fill="none" stroke="{INK}" stroke-width="3"/>')
body.append(f'<path d="M{num(px(0))} {num(py(0))} L{num(px(span))} {num(py(rent_at(span)))}" fill="none" stroke="{ORANGE}" stroke-width="4"/>')
cx = px(cross["tokens"])
body.append(f'<path d="M{num(cx)} {base}V{top}" stroke="{GREY}" stroke-width="3" stroke-dasharray="{DASH}"/>')
body.append(f'<circle cx="{num(cx)}" cy="{num(py(cross["own"]))}" r="8" fill="{GREEN}" stroke="{INK}" stroke-width="2.5"/>')
body.append(text(cx, top - 10, f"{billions(cross['tokens'])} billion", "middle", INK, bold=True))
body.append(text(cx - 12, base - 14, "rent costs less", "end", INK, bold=True))
body.append(text(cx + 12, base - 14, "own costs less", "start", INK, bold=True))
body.append(text(px(0.85 * span), py(rent_at(0.85 * span)) - 18, "rent", "end", INK, bold=True))
body.append(text(x1, py(own_at(steps)) + 34, "own", "end", INK, bold=True))
body.append(f'<path d="M{x0} {base}H{x1}" stroke="{GREY}" stroke-width="2"/>')
for tick in range(0, int(span / 1e9) + 1, 100):
body.append(text(px(tick * 1e9), base + 30, str(tick), "middle", GREY, size=20))
body.append(text((x0 + x1) / 2, base + 62, "billion tokens a month", "middle", GREY))
desc = (f"Renting (a rising orange line) and owning (black steps) over 0 to {billions(span)} billion tokens a month, in dollars a month, with the traffic spread over 24 hours a day. "
f"Renting starts at nothing and rises {money(per_reply * 1e9 / rvo.TOKENS_PER_REPLY)} for each billion tokens. Owning starts at {money(own_at(1))} (the people, "
f"{rvo.cp.gpu_word(1 + rvo.SPARE)}) and steps up {money(gpu_month)} for each GPU added, every {billions(step)} billion tokens or so. The lines cross at "
f"{billions(cross['tokens'])} billion tokens a month, with {cross['gpus']} GPUs paid for there. At {billions(TOKENS)} billion, renting costs {money(now['rent'])} "
f"and owning {money(now['own'])}.")
return frame(WIDTH, 380, f"Rent or own: the lines cross at {billions(cross['tokens'])} billion tokens a month", desc, body)
# --- figure 2: what a fixed set of GPUs costs per million tokens -----------------------------------------------------------------------------
def fig_busy() -> str:
now = rvo.case(TOKENS, 24)
fleet = float(now["own"])
capacity = now["gpus"] * rvo.REPLIES_PER_GPU_SECOND * rvo.SECONDS_PER_MONTH * rvo.TOKENS_PER_REPLY / 1e6 # millions of tokens a month at 100% busy
rent = float(now["rent_per_million"])
own_at = lambda busy: fleet / (busy * capacity)
equal = fleet / (rent * capacity) # the busy share at which this fleet costs what renting costs
ceiling = rvo.busy_share((now["gpus"] - rvo.SPARE) * rvo.rates(24)[2], now["gpus"]) # the most these GPUs reach: the planned ones at their planned load, the spare idle
x0, x1, top, base, lo, ymax = 96, 496, 60, 300, 0.10, 0.60
px = lambda busy: x0 + (busy - lo) / (1 - lo) * (x1 - x0)
py = lambda cost: base - cost / ymax * (base - top)
first = math.ceil(fleet / (ymax * capacity) * 100) / 100
body = [text(20, 36, "dollars per million tokens", color=GREY)]
for tick in (0, 0.2, 0.4, 0.6):
body.append(f'<path d="M{x0} {num(py(tick))}H{x1}" stroke="{FILL}" stroke-width="2"/>')
body.append(text(x0 - 8, py(tick) + 7, f"{tick:.2f}", "end", GREY, size=20))
busy_points = [first + k / 100 for k in range(0, math.floor((ceiling - first) * 100) + 1)] + [ceiling]
curve = "M" + " L".join(f"{num(px(b))} {num(py(own_at(b)))}" for b in busy_points)
body.append(f'<path d="{curve}" fill="none" stroke="{INK}" stroke-width="3"/>')
body.append(f'<path d="M{x0} {num(py(rent))}H{x1}" stroke="{ORANGE}" stroke-width="4"/>')
body.append(text(x1, py(rent) - 14, f"rent ${rent:.4f}", "end", INK, bold=True))
body.append(text(px(first) + 14, py(own_at(first)) + 30, f"own, {now['gpus']} GPUs", "start", INK, bold=True))
body.append(f'<path d="M{num(px(equal))} {base}V{top}" stroke="{GREY}" stroke-width="3" stroke-dasharray="{DASH}"/>')
body.append(text(px(equal), top - 10, f"{rvo.percent(equal)} busy", "middle", INK, bold=True))
body.append(f'<circle cx="{num(px(now["busy"]))}" cy="{num(py(own_at(now["busy"])))}" r="8" fill="{GREEN}" stroke="{INK}" stroke-width="2.5"/>')
body.append(text(px(now["busy"]) - 14, py(own_at(now["busy"])) + 32, f"${own_at(now['busy']):.4f}", "end", INK, halo=True))
body.append(f'<circle cx="{num(px(ceiling))}" cy="{num(py(own_at(ceiling)))}" r="6" fill="#ffffff" stroke="{INK}" stroke-width="3"/>')
body.append(text(px(ceiling) + 14, py(own_at(ceiling)) + 32, f"at most {rvo.percent(ceiling)}", "start", INK))
body.append(f'<path d="M{x0} {base}H{x1}" stroke="{GREY}" stroke-width="2"/>')
for tick in (0.2, 0.4, 0.6, 0.8, 1.0):
body.append(text(px(tick), base + 30, f"{round(tick * 100)}%", "middle", GREY, size=20))
body.append(text((x0 + x1) / 2, base + 62, "how busy the GPUs are", "middle", GREY))
desc = (f"Dollars per million tokens against how busy the GPUs are, from {round(lo * 100)}% to 100%, for a fixed set of {now['gpus']} GPUs that costs {money(fleet)} a month. "
f"The black curve is the owning cost per million tokens; it falls as the busy share rises. The orange line is the rent price, ${rent:.4f}. The curve passes under the line at "
f"{rvo.percent(equal)} busy. At {rvo.percent(now['busy'])} busy, the case of the lesson, owning is ${own_at(now['busy']):.4f}. The curve ends at {rvo.percent(ceiling)} busy, "
f"the most that {now['gpus'] - rvo.SPARE} planned GPUs and a spare can reach.")
return frame(WIDTH, 380, f"Owning costs less per million tokens above {rvo.percent(equal)} busy", desc, body)
# --- figure 3: the same tokens, in fewer hours a day -----------------------------------------------------------------------------------------
HOURS_SHOWN = (24, 18, 12, 8, 4)
def fig_peaky() -> str:
rows = [(h, rvo.case(TOKENS, h)) for h in HOURS_SHOWN]
rent = float(rows[0][1]["rent"])
longest = max(float(c["own"]) for _, c in rows)
x0, span = 24, 250
scale = span / longest
body, y = [text(x0, 30, f"dashed mark: renting, {money(rent)} a month", color=GREY)], 76
for h, c in rows:
body.append(text(x0, y, f"{h} hours a day: {c['gpus']} GPUs, {rvo.percent(c['busy'])} busy", bold=True))
own = float(c["own"])
cheaper = own < rent
body.append(f'<rect x="{x0}" y="{y + 12}" width="{num(own * scale)}" height="30" fill="{PANEL}" stroke="{GREEN if cheaper else ORANGE}" stroke-width="4"/>')
body.append(f'<path d="M{num(x0 + rent * scale)} {y + 6}V{y + 48}" stroke="{INK}" stroke-width="3" stroke-dasharray="6 5"/>')
body.append(text(x0 + max(own, rent) * scale + 10, y + 36, f"{money(own)}, " + ("cheaper" if cheaper else "dearer")))
y += 68
facts = "; ".join(f"{h} hours: {c['gpus']} GPUs, {money(c['own'])}, busy {rvo.percent(c['busy'])}" for h, c in rows)
desc = (f"The same {billions(TOKENS)} billion tokens a month with all the traffic in {', '.join(str(h) for h in HOURS_SHOWN)} hours a day. One bar for what owning costs a month "
f"against a dashed line at the rent, {money(rent)}. {facts}. Owning is cheaper only when the traffic fills most of the day.")
return frame(WIDTH, y - 12, "The same tokens cost more to own the fewer hours they arrive in", desc, body)
# --- figure 4: what the people cost does to the crossing -------------------------------------------------------------------------------------
PEOPLE_SHOWN = (0, 5_000, 10_000, 20_000, 30_000)
def fig_people() -> str:
rows = [(p, rvo.break_even(24, people=p)) for p in PEOPLE_SHOWN]
longest = max(x["tokens"] for _, x in rows)
x0, span = 24, 300
scale = span / longest
body, y = [], 30
for people, x in rows:
body.append(text(x0, y, f"people {money(people)} a month: {x['gpus']} GPUs", bold=True))
body.append(f'<rect x="{x0}" y="{y + 12}" width="{num(x["tokens"] * scale)}" height="30" fill="{PANEL}" stroke="{INK}" stroke-width="2"/>')
body.append(text(x0 + x["tokens"] * scale + 10, y + 36, f"{rvo.tokens_words(x['tokens']).replace(' billion', '')} billion"))
y += 68
facts = "; ".join(f"{money(p)}: {rvo.tokens_words(x['tokens'])} tokens with {x['gpus']} GPUs" for p, x in rows)
desc = (f"Where owning first costs no more than renting, by what the people cost a month, with the traffic spread over 24 hours a day. One bar each, longer when the people cost more. {facts}.")
return frame(WIDTH, y - 10, "The more the people cost, the later owning catches up with renting", desc, body)
FIGURES = {
"rent-or-own.svg": fig_lines,
"cost-per-million.svg": fig_busy,
"same-tokens-fewer-hours.svg": fig_peaky,
"people-cost.svg": fig_people,
}
def main() -> int:
stale = []
for name, draw in FIGURES.items():
path, svg = FIGS / name, draw()
if "--check" in sys.argv:
if not path.exists() or path.read_text(encoding="utf-8") != svg:
stale.append(name)
else:
FIGS.mkdir(exist_ok=True)
path.write_text(svg, encoding="utf-8")
if "--check" in sys.argv:
print("figures differ from rent_vs_own.py: " + ", ".join(stale) if stale else "figures match rent_vs_own.py")
return 1 if stale else 0
print("wrote " + ", ".join(FIGURES))
return 0
if __name__ == "__main__":
sys.exit(main())Show code
"""Rent a model API or run your own: each way's bill for a month, and the monthly token count where they cross.
Python standard library only. Run `python3 show_numbers.py` to print every number the lesson quotes, and `python3 trace.py` to follow one case step by step.
The model: a service sends a number of tokens a month, all of it in the same hours every day. Renting bills each token at its price (lesson 7's bill). Owning pays
for enough GPUs for the busiest hour (lesson 29's plan) plus a spare, for every hour of the month, plus a fixed monthly cost for the people who run them.
Prices are dollars; token prices are per million tokens.
"""
from __future__ import annotations
import importlib.util
import math
import statistics
import sys
from decimal import ROUND_HALF_UP, Decimal
from pathlib import Path
sys.dont_write_bytecode = True # loading the sibling lessons' files below must not leave a __pycache__ in their folders
LESSONS = Path(__file__).resolve().parents[2]
def _load(name: str, folder: str, filename: str):
"""A sibling lesson folder is not a package, so its file is loaded by path."""
spec = importlib.util.spec_from_file_location(name, LESSONS / folder / "code" / filename)
module = importlib.util.module_from_spec(spec)
sys.modules[spec.name] = module # dataclasses looks the module up by name
spec.loader.exec_module(module)
return module
bill = _load("bill_lesson_7", "why-ai-bill-is-high", "bill.py")
cp = _load("capacity_plan_lesson_29", "how-many-gpus-for-1000-users", "capacity_plan.py")
# --- the inputs: round toy figures, not quotes -----------------------------------------------------------------------------------------------
MIX = cp.OPENING_MIX # lesson 29's usual chats
TARGET_MS = cp.TARGET_MS # and its target: the p99 gap between words stays under 50 ms
HEADROOM = cp.HEADROOM # and its 20% in hand
DAYS = bill.DAYS_PER_MONTH # lesson 7's month: 30 days
HOURS_PER_MONTH = 24 * DAYS
SECONDS_PER_MONTH = HOURS_PER_MONTH * 3600
_chats = cp.chats_of(MIX)
TOKENS_IN = round(statistics.fmean(_chats["prompts"])) # 1,882: the prompt each reply is sent with, on average
TOKENS_OUT = round(statistics.fmean(_chats["replies"])) # 157: the reply itself
TOKENS_PER_REPLY = TOKENS_IN + TOKENS_OUT
PRICE_IN = Decimal("0.10") # dollars per million input tokens: the cheapest row on the price page
PRICE_OUT = Decimal("0.50") # dollars per million output tokens
GPU_HOUR = Decimal("3.00") # dollars to keep one GPU for one hour
PEOPLE = 10_000 # dollars a month for the people who run the GPUs
SPARE = 1 # lesson 29's spare
MAX_GPUS = 5000 # the search for a break-even stops here: past this the toy has nothing more to say
DESIGN = cp.plan(1, 100, 3600, MIX, TARGET_MS) # lesson 29's plan, for the numbers every GPU count here is divided by
REPLIES_PER_GPU_SECOND = DESIGN["full"] / DESIGN["stay"] # replies a second one GPU finishes when it is as full as its tighter limit allows
OPENING = {"tokens_b": 300, "hours": 24, "people": PEOPLE, "price_in": 0.10, "price_out": 0.50, "gpu_hour": 3.0, "spare": SPARE}
TOKENS_B = [1, 2, 5, 10, 20, 30, 50, 75, 100, 150, 200, 300, 400, 500, 750, 1000, 2000] # the widget's monthly tokens, in billions
PEOPLE_RANGE = (0, 40_000, 1000) # lowest, highest, step
HOURS_RANGE = (1, 24, 1)
PRICES_IN = [0.10, 0.50, 1.0, 2.0, 4.0, 10.0]
PRICES_OUT = [0.50, 2.0, 5.0, 10.0, 20.0, 50.0]
GPU_HOURS = [1.0, 1.5, 2.0, 2.5, 3.0, 4.0, 5.0, 6.0, 8.0]
SPARES = [0, 1, 2]
# --- the six steps ---------------------------------------------------------------------------------------------------------------------------
def replies_for(tokens):
return math.floor(tokens / TOKENS_PER_REPLY + 0.5)
def monthly_rent(replies, price_in=PRICE_IN, price_out=PRICE_OUT):
if replies < 1:
return Decimal(0)
return bill.bill(bill.Chat(system=0, message=TOKENS_IN, reply=TOKENS_OUT, turns=replies, keep=0), price_in, price_out).total
def gpus_to_pay_for(replies, hours, spare=SPARE):
return cp.plan(1, 100, replies / (DAYS * hours), MIX, TARGET_MS)["n"] + spare
def monthly_own(gpus, gpu_hour=GPU_HOUR, people=PEOPLE):
return gpus * HOURS_PER_MONTH * gpu_hour + people
def busy_share(replies, gpus):
return replies / (gpus * REPLIES_PER_GPU_SECOND * SECONDS_PER_MONTH)
def rates(hours, price_in=PRICE_IN, price_out=PRICE_OUT, gpu_hour=GPU_HOUR):
"""What the owning and renting sums are made of: dollars the rent charges for one reply, dollars one GPU costs for the month, and the replies one planned GPU
carries in a month when all the traffic falls in `hours` hours a day (HEADROOM of what it finishes at its limit)."""
per_reply = (TOKENS_IN * round(price_in * 100) + TOKENS_OUT * round(price_out * 100)) / 10**8
gpu_month = round(gpu_hour * 100) * HOURS_PER_MONTH / 100
carry = HEADROOM * REPLIES_PER_GPU_SECOND * DAYS * hours * 3600
return per_reply, gpu_month, carry
def break_even(hours, price_in=PRICE_IN, price_out=PRICE_OUT, gpu_hour=GPU_HOUR, people=PEOPLE, spare=SPARE):
"""The monthly replies and tokens at which owning first costs no more than renting, with the GPUs paid for there, or None when that takes over MAX_GPUS GPUs.
With n planned GPUs the owning bill is the same for every reply count up to n * carry, while the rent keeps rising. So the first n whose bill is no more than the
rent at n * carry holds the crossing, and the crossing is where the rent has risen to that bill.
"""
per_reply, gpu_month, carry = rates(hours, price_in, price_out, gpu_hour)
for n in range(1, MAX_GPUS + 1):
own = people + (n + spare) * gpu_month
if own <= per_reply * n * carry:
return {"replies": own / per_reply, "tokens": own / per_reply * TOKENS_PER_REPLY, "gpus": n + spare, "own": own}
return None
# --- around the steps ------------------------------------------------------------------------------------------------------------------------
def dollars(x) -> int:
"""Whole dollars, halves up."""
return int(Decimal(x).quantize(Decimal(1), rounding=ROUND_HALF_UP))
def case(tokens, hours, people=PEOPLE, price_in=PRICE_IN, price_out=PRICE_OUT, gpu_hour=GPU_HOUR, spare=SPARE) -> dict:
"""Both bills for one month, the GPUs and the busy share, from the six steps."""
replies = replies_for(tokens)
rent = monthly_rent(replies, price_in, price_out)
gpus = gpus_to_pay_for(replies, hours, spare)
own = monthly_own(gpus, gpu_hour, people)
return {"tokens": tokens, "replies": replies, "rent": rent, "gpus": gpus, "own": own, "busy": busy_share(replies, gpus),
"own_per_million": own / Decimal(tokens) * bill.PER, "rent_per_million": rent / Decimal(tokens) * bill.PER}
def commas(n: int) -> str:
return f"{n:,}"
def tokens_words(tokens: float) -> str:
"""A token count to three figures, in millions, billions or trillions."""
unit, name = (1e12, "trillion") if tokens >= 1e12 else (1e9, "billion") if tokens >= 1e9 else (1e6, "million")
v = tokens / unit
places = 0 if v >= 100 else 1 if v >= 10 else 2
return f"{math.floor(v * 10**places + 0.5) / 10**places:.{places}f} {name}"
def percent(x: float) -> str:
return f"{math.floor(x * 100 + 0.5)}%"
# --- the widget: the words and the chart for one setting of its controls ---------------------------------------------------------------------
W, H, PAD = 300, 150, 4 # the chart's drawing box; the lines use all of it, and the words under it say what the sides are
MAX_STEPS = 120 # the chart draws at most this many steps of the owning line, grouping GPUs when there are more
def fix1(x: float) -> float:
return math.floor(x * 10 + 0.5) / 10
def spare_words(spare: int) -> str:
return "no spare" if spare == 0 else f"{spare} spare" if spare == 1 else f"{spare} spares"
def chart(hours, tokens, cross, people, price_in, price_out, gpu_hour, spare, rent_now, own_now) -> dict:
"""The two lines of the chart, as path text for an SVG, over the monthly tokens from 0 to a little past both the reader's setting and the crossing, and the
heights of the two bills at the reader's setting."""
per_reply, gpu_month, carry = rates(hours, price_in, price_out, gpu_hour)
span = 1.25 * max(tokens, cross["tokens"] if cross else 0)
step = carry * TOKENS_PER_REPLY # tokens one planned GPU carries in a month
steps = math.ceil(span / step)
stride = max(1, math.ceil(steps / MAX_STEPS))
own_at = lambda n: people + (n + spare) * gpu_month
top = max(per_reply * span / TOKENS_PER_REPLY, own_at(steps))
x = lambda t: fix1(PAD + (W - 2 * PAD) * t / span)
y = lambda cost: fix1(H - PAD - (H - 2 * PAD) * cost / top)
points = []
for k in range(0, steps, stride):
n = min(k + stride, steps)
points += [(x(k * step), y(own_at(n))), (x(min(n * step, span)), y(own_at(n)))]
joined = lambda pts: "M" + " L".join(f"{px:.1f},{py:.1f}" for px, py in pts)
return {"rent": joined([(x(0), y(0)), (x(span), y(per_reply * span / TOKENS_PER_REPLY))]), "own": joined(points),
"cross": None if cross is None else f"{x(cross['tokens']):.1f}", "mark": f"{x(tokens):.1f}", "dots": [f"{y(rent_now):.1f}", f"{y(own_now):.1f}"],
"caption": f"Across: 0 to {tokens_words(span)} tokens a month. Up: 0 to ${commas(dollars(top))} a month."}
def describe(tokens_b, hours, people, price_in, price_out, gpu_hour, spare) -> dict:
"""Everything the widget shows for one setting of its controls: three lines of words and the two lines of the chart."""
price_in, price_out, gpu_hour = Decimal(str(price_in)), Decimal(str(price_out)), Decimal(str(gpu_hour))
tokens = tokens_b * 10**9
now = case(tokens, hours, people, price_in, price_out, gpu_hour, spare)
cross = break_even(hours, price_in, price_out, gpu_hour, people, spare)
rent, own = dollars(now["rent"]), dollars(now["own"])
cheaper = "Owning costs less." if now["own"] < now["rent"] else "Renting costs less." if now["rent"] < now["own"] else "They cost the same."
return {"headline": f"Below about {tokens_words(cross['tokens'])} tokens a month, renting costs less." if cross else "At these settings owning never catches up with renting.",
"verdict": f"At {commas(tokens_b)} billion tokens a month, renting costs ${commas(rent)} and owning ${commas(own)}. {cheaper}",
"detail": f"Owning takes {cp.gpu_word(now['gpus'])} ({spare_words(spare)}), busy {percent(now['busy'])} of the time.",
"chart": chart(hours, tokens, cross, people, float(price_in), float(price_out), float(gpu_hour), spare, float(now["rent"]), float(now["own"]))}Show code
"""Print every number the lesson quotes. Run it from this folder: python3 show_numbers.py"""
from decimal import Decimal
from rent_vs_own import (DAYS, GPU_HOUR, HEADROOM, HOURS_PER_MONTH, OPENING, PEOPLE, PRICE_IN, PRICE_OUT, REPLIES_PER_GPU_SECOND, SECONDS_PER_MONTH, SPARE,
TOKENS_IN, TOKENS_OUT, TOKENS_PER_REPLY, break_even, case, commas, describe, dollars, percent, rates, tokens_words)
T = 300 * 10**9
print("per reply", TOKENS_IN, TOKENS_OUT, TOKENS_PER_REPLY)
print("replies per GPU second at the limit", round(REPLIES_PER_GPU_SECOND, 4), "headroom", HEADROOM)
per_reply, gpu_month, carry = rates(24)
print(f"one planned GPU carries {commas(round(carry))} replies = {tokens_words(carry * TOKENS_PER_REPLY)} tokens a month at 24 hours a day")
print(f"rent price of that: ${per_reply * carry:,.0f}; a GPU costs ${gpu_month:,.0f} a month; ratio {gpu_month / (per_reply * carry):.3f}")
print(f"owning can only win if each GPU is busy more than {gpu_month / (per_reply * carry / HEADROOM):.1%}")
print(f"hours a day below which a planned GPU rents for less than it costs: {24 * gpu_month / (per_reply * carry):.2f}")
print(f"rent per million tokens: ${float(PRICE_IN * TOKENS_IN + PRICE_OUT * TOKENS_OUT) / TOKENS_PER_REPLY:.4f}")
print("\nthe opening case by hours a day")
for h in (24, 18, 16, 12, 8, 6, 4):
c = case(T, h)
x = break_even(h)
print(f"{h:>2} h replies {commas(c['replies'])} rent ${commas(dollars(c['rent']))} gpus {c['gpus']} own ${commas(dollars(c['own']))} busy {c['busy']:.4f} {percent(c['busy'])} "
f"own/rent {float(c['own'] / c['rent']):.2f} per M own {float(c['own_per_million']):.4f} rent {float(c['rent_per_million']):.4f} "
f"cross {tokens_words(x['tokens']) + ' gpus ' + str(x['gpus']) + ' own $' + commas(dollars(x['own'])) if x else None}")
print("\nbusy share at the crossing, 24 h")
x = break_even(24)
from rent_vs_own import busy_share
print(x, busy_share(x["replies"], x["gpus"]), percent(busy_share(x["replies"], x["gpus"])))
c250 = case(250 * 10**9, 24)
c400 = case(400 * 10**9, 24)
for c in (c250, c400):
print(commas(c["tokens"]), "rent", dollars(c["rent"]), "gpus", c["gpus"], "own", dollars(c["own"]), "busy", percent(c["busy"]))
print("\npeople cost, 24 h")
for p in (0, 5_000, 10_000, 20_000, 30_000):
x = break_even(24, people=p)
print(p, tokens_words(x["tokens"]), x["gpus"], dollars(x["own"]), percent(busy_share(x["replies"], x["gpus"])))
print("\nspare, 24 h")
for s in (0, 1, 2):
x = break_even(24, spare=s)
print("spare", s, tokens_words(x["tokens"]), x["gpus"])
print("\nprices")
for pi, po in ((Decimal("0.10"), Decimal("0.50")), (Decimal(2), Decimal(10))):
x = break_even(24, pi, po)
c = case(T, 24, price_in=pi, price_out=po)
print(pi, po, tokens_words(x["tokens"]), x["gpus"], "at 300B rent", dollars(c["rent"]), "own", dollars(c["own"]))
for gh in (Decimal("2.0"), Decimal("4.0")):
x = break_even(24, gpu_hour=gh)
print("gpu hour", gh, tokens_words(x["tokens"]) if x else None, x["gpus"] if x else None)
print("\nthe opening widget state")
d = describe(**OPENING)
print(d["headline"]); print(d["verdict"]); print(d["detail"]); print(d["chart"]["caption"])
print("\n4 h widget state")
d = describe(**{**OPENING, "hours": 4})
print(d["headline"]); print(d["verdict"]); print(d["detail"])
print(DAYS, HOURS_PER_MONTH, SECONDS_PER_MONTH, GPU_HOUR, PEOPLE, SPARE)Show code
"""Follow one month of traffic through the six steps. Run it from this folder: python3 trace.py
It uses the lesson's own functions and adds no rules of its own: each step calls the function the table in the lesson shows, and this file only prints what it
returns. The prices are the lesson's round toy prices, not a quote.
"""
from rent_vs_own import (DAYS, GPU_HOUR, HOURS_PER_MONTH, PEOPLE, PRICE_IN, PRICE_OUT, REPLIES_PER_GPU_SECOND, SECONDS_PER_MONTH, SPARE, TOKENS_IN, TOKENS_OUT,
break_even, busy_share, dollars, gpus_to_pay_for, monthly_own, monthly_rent, replies_for, tokens_words)
TOKENS = 300 * 10**9 # tokens sent in a month, input and output together
HOURS = 24 # the hours of each day in which all the traffic falls
replies = replies_for(TOKENS)
print(f"1. demand {TOKENS:,} tokens a month = {replies:,} replies of {TOKENS_IN:,} in + {TOKENS_OUT} out")
rent = monthly_rent(replies)
print(f"2. rent {replies:,} replies at ${PRICE_IN} in and ${PRICE_OUT} out per million tokens = ${dollars(rent):,} a month")
gpus = gpus_to_pay_for(replies, HOURS)
per_second = replies / (DAYS * HOURS * 3600)
print(f"3. GPUs traffic in {HOURS} hours a day = {per_second:.1f} replies a second at the peak; lesson 29's plan: {gpus - SPARE} GPUs + {SPARE} spare = {gpus}")
own = monthly_own(gpus)
print(f"4. own {gpus} GPUs x {HOURS_PER_MONTH} hours x ${GPU_HOUR} = ${dollars(gpus * HOURS_PER_MONTH * GPU_HOUR):,}, plus ${PEOPLE:,} for people = ${dollars(own):,} a month")
busy = busy_share(replies, gpus)
print(f"5. busy {replies:,} replies / ({gpus} GPUs x {REPLIES_PER_GPU_SECOND:.2f} a second x {SECONDS_PER_MONTH:,} seconds) = {busy:.1%}")
print(f" per million tokens: owning ${own / (TOKENS // 10**6):.4f}, renting ${rent / (TOKENS // 10**6):.4f}")
cross = break_even(HOURS)
if cross:
print(f"6. cross owning first costs no more than renting at {tokens_words(cross['tokens'])} tokens a month, with {cross['gpus']} GPUs paid for there")
print(f" at {tokens_words(TOKENS)}: owning is ${dollars(rent - own):,} {'cheaper' if own < rent else 'dearer'} than renting")
else:
print(f"6. cross no crossing in {HOURS} hours a day: owning stays dearer however many tokens are sent")
print(f" at {tokens_words(TOKENS)}: owning is ${dollars(own - rent):,} {'dearer' if own > rent else 'cheaper'} than renting")
print("\nthe same tokens, with all the traffic in fewer hours a day")
for hours in (18, 12, 6):
gpus = gpus_to_pay_for(replies, hours)
cross = break_even(hours)
print(f" {hours:>2} hours: {gpus} GPUs, owning ${dollars(monthly_own(gpus)):,} against renting ${dollars(rent):,}, busy {busy_share(replies, gpus):.1%}, "
f"crossing {tokens_words(cross['tokens']) + ' tokens' if cross else 'none'}")
print("\nthe same traffic, with other costs for the people (traffic all day)")
for people in (0, 10_000, 20_000):
cross = break_even(24, people=people)
print(f" ${people:>6,} a month: crossing at {tokens_words(cross['tokens'])} tokens, with {cross['gpus']} GPUs")Show code
"""Guards for rent_vs_own.py: the tokens a month, both bills, the GPU count, the busy share and the break-even, each checked against a sum made by hand or an
independent route."""
import math
import re
import subprocess
import sys
import textwrap
from decimal import Decimal
from pathlib import Path
import pytest
import make_figs
import rent_vs_own as r
from rent_vs_own import cp
CODE = Path(__file__).resolve().parent
LESSON = CODE.parent
WEB_ADDRESS = re.compile("ht" + r"tps?://") # written in two pieces so that this file holds none either
def test_a_month_of_tokens_becomes_replies_and_the_rent_is_the_per_token_bill():
assert (r.TOKENS_IN, r.TOKENS_OUT, r.TOKENS_PER_REPLY) == (1882, 157, 2039) # lesson 29's usual chats, to the whole token
assert r.replies_for(300 * 10**9) == 147_130_947 and r.replies_for(0) == 0 and r.replies_for(1019) == 0 and r.replies_for(1020) == 1
for replies in (1, 1000, 147_130_947):
for price_in, price_out in ((Decimal("0.10"), Decimal("0.50")), (Decimal(2), Decimal(10))):
by_hand = (replies * 1882 * price_in + replies * 157 * price_out) / 1_000_000 # tokens times price, per million
assert r.monthly_rent(replies, price_in, price_out) == by_hand
assert r.dollars(r.monthly_rent(147_130_947)) == 39_240 and r.monthly_rent(0) == 0
def test_the_gpu_count_is_lessons_29_plan_plus_the_spare():
month = r.SECONDS_PER_MONTH
assert r.gpus_to_pay_for(20 * month, 24) == cp.plan(1000, 60, 120, r.MIX, r.TARGET_MS)["n"] + 1 == 5 # lesson 29's 1,000 users at the busiest hour's pace all day
for hours in (24, 12, 4): # one GPU carries 80% of what it finishes, and a month of traffic hours is 30 * hours * 3600 s
carry = 0.8 * r.REPLIES_PER_GPU_SECOND * 30 * hours * 3600
for n in (1, 2, 5, 9):
assert r.gpus_to_pay_for(math.floor(n * carry), hours) == n + 1 and r.gpus_to_pay_for(math.floor(n * carry) + 2, hours) == n + 2
assert r.gpus_to_pay_for(math.floor(n * carry), hours, spare=0) == n
def test_the_owning_bill_is_hours_times_price_plus_people_and_the_busy_share_counts_every_paid_gpu():
assert r.monthly_own(12) == 12 * 720 * 3 + 10_000 == 35_920 and r.monthly_own(1, Decimal("2.5"), 0) == 1800
for n in (1, 3, 12): # n planned GPUs at their planned load, plus the spare, are busy 80% * n / (n + spare) of the month
carry = 0.8 * r.REPLIES_PER_GPU_SECOND * r.SECONDS_PER_MONTH
assert r.busy_share(n * carry, n + 1) == pytest.approx(0.8 * n / (n + 1))
assert r.busy_share(0, 5) == 0 and r.busy_share(r.REPLIES_PER_GPU_SECOND * r.SECONDS_PER_MONTH, 1) == pytest.approx(1)
now = r.case(300 * 10**9, 24)
assert (now["gpus"], r.dollars(now["rent"]), r.dollars(now["own"]), round(now["busy"], 3)) == (12, 39_240, 35_920, 0.677)
assert now["own_per_million"] == now["own"] / 300_000 and now["rent_per_million"] == now["rent"] / 300_000
SETTINGS = [dict(hours=h, people=p, price_in=pi, price_out=po) for h in (24, 18) for p in (0, 10_000, 20_000) for pi, po in ((Decimal("0.10"), Decimal("0.50")), (Decimal(2), Decimal(10)))]
def test_break_even_is_the_first_token_count_where_owning_costs_no_more_than_renting():
def cheaper_to_own(replies, s):
return r.monthly_own(r.gpus_to_pay_for(replies, s["hours"]), people=s["people"]) <= r.monthly_rent(replies, s["price_in"], s["price_out"])
for s in SETTINGS:
found = r.break_even(s["hours"], s["price_in"], s["price_out"], people=s["people"])
top = math.ceil(found["replies"])
assert cheaper_to_own(top, s) and found["gpus"] == r.gpus_to_pay_for(top, s["hours"]) # at the crossing owning is no dearer, with these GPUs
assert not cheaper_to_own(top - 3, s) or found["replies"] < 3 # and a hair below it, renting is cheaper
assert not any(cheaper_to_own(math.ceil(top * k / 400), s) for k in range(1, 400)), s # nowhere below: no token count there favours owning
assert r.break_even(4) is None and r.break_even(13) is None and r.break_even(14)["gpus"] == 132 # a load too peaky for its GPUs never crosses
def test_people_cost_moves_the_break_even_by_the_formula_the_lesson_gives():
seen = [r.break_even(24, people=p)["tokens"] for p in (0, 10_000, 20_000)]
assert seen[0] < seen[1] < seen[2] and r.dollars(r.break_even(24)["tokens"] / 1e9) == 225
per_reply, gpu_month, carry = r.rates(24)
for p, exact in zip((0, 10_000, 20_000), seen): # (people + spare * GPU month) / (rent per reply - GPU month / carry), in tokens: within one GPU's worth
smooth = (p + r.SPARE * gpu_month) / (per_reply - gpu_month / carry) * r.TOKENS_PER_REPLY
assert abs(smooth - exact) <= carry * r.TOKENS_PER_REPLY, (p, smooth, exact)
def lesson_text() -> str:
return (LESSON / "lesson.md").read_text(encoding="utf-8")
def test_each_file_and_line_the_lesson_cites_holds_the_code_it_quotes():
prose = lesson_text()
cites = re.findall(r"`([\w]+\.py):(\d+)`<br>`(.+?)` \|", prose)
assert [int(line) for _, line, _ in cites] == [68, 74, 78, 82, 86, 107] # the six steps, one row each
for name, line, quoted in cites:
assert (CODE / name).read_text(encoding="utf-8").splitlines()[int(line) - 1].strip() == quoted, (name, line)
file_lines = (CODE / "rent_vs_own.py").read_text(encoding="utf-8").splitlines()
assert file_lines[104].strip() == "for n in range(1, MAX_GPUS + 1):" and "The `for` loop on line 105" in prose # the row for step 6 sends the reader to line 105
excerpt = textwrap.dedent("\n".join(file_lines[104:108]))
assert f"```python\n{excerpt}\n```" in prose # and Go deeper shows those four lines as they are
def test_the_trace_prints_the_runs_block_and_the_one_line_edit_gives_the_hidden_answer():
prose = lesson_text()
blocks = re.findall(r"```text\n(.*?)```", prose, re.S)
assert len(blocks) == 2
source = (CODE / "trace.py").read_text(encoding="utf-8")
assert source.splitlines()[9].startswith("HOURS = 24 ") and "change line 10 to `HOURS = 12`" in prose # the Your turn line is the line it names
def run(code):
return subprocess.run([sys.executable, "-c", code], cwd=CODE, capture_output=True, text=True, check=True, env={"PYTHONDONTWRITEBYTECODE": "1"}).stdout
assert run(source) == blocks[0]
assert run(source.replace("HOURS = 24 ", "HOURS = 12 ", 1)).startswith(blocks[1]) # the top of the output, as the hidden answer shows it
def test_every_number_in_the_lesson_and_figures_comes_from_the_code():
"""The prose, the trace and the four figures quote numbers by hand; a change to a constant or to lesson 29's planner must show up here, not in a reader's check."""
prose, figures = lesson_text(), {path.name: path.read_text(encoding="utf-8") for path in sorted((LESSON / "figs").glob("*.svg"))}
assert set(figures) == set(re.findall(r"\]\(figs/([\w-]+\.svg)\)", prose)) == set(make_figs.FIGURES) and len(figures) == 4
for name, draw in make_figs.FIGURES.items(): # the files on disk are what make_figs.py draws now
assert figures[name] == draw(), name
D = lambda x: "$" + r.commas(r.dollars(x))
B, P = r.tokens_words, r.percent
W = lambda tokens: B(tokens).removesuffix(" billion") # "225" from "225 billion"
T = 300 * 10**9
per_reply, gpu_month, carry = r.rates(24)
now, x24, c12, c4 = r.case(T, 24), r.break_even(24), r.case(T, 12), r.case(T, 4)
rent_pm, own_pm = float(now["rent_per_million"]), float(now["own_per_million"])
by = {h: (r.case(T, h), r.break_even(h)) for h in (24, 18, 16, 12, 8, 4)}
people = {p: r.break_even(24, people=p) for p in (0, 5_000, 10_000, 20_000, 30_000)}
spare = {s: r.break_even(24, spare=s) for s in (0, 1, 2)}
gpu_hour = {g: r.break_even(24, gpu_hour=Decimal(g)) for g in ("2.0", "4.0")}
dear = r.break_even(24, Decimal(2), Decimal(10))
flat_out = r.REPLIES_PER_GPU_SECOND * r.SECONDS_PER_MONTH
smooth = (r.PEOPLE + r.SPARE * gpu_month) / (per_reply - gpu_month / carry)
floor = gpu_month / (flat_out * per_reply) # busy share below which a GPU costs more than the rent for its replies
busy_cross = now["busy"] * own_pm / rent_pm # the 12 GPUs' busy share where owning per million tokens equals renting
shifts = [(people[b]["tokens"] - people[a]["tokens"]) / 1e9 for a, b in ((0, 10_000), (10_000, 20_000), (20_000, 30_000))] # tokens moved per $10,000 of people, in billions
nearest5 = lambda x: round(x / 5) * 5
assert [nearest5(x) for x in shifts[:2]] == [175, 175] and nearest5(min(shifts)) == 160 and nearest5(max(shifts)) == 175 # "about 175" to $20,000, "about 160 to 175" over the table
assert r.break_even(13) is None and r.break_even(24, Decimal("0.05"), Decimal("0.25")) is None # "13 hours or fewer", "half the price"
assert 0.69 <= float(r.GPU_HOUR) <= 6.99 and 0.10 <= float(r.PRICE_IN) <= 15 and 0.50 <= float(r.PRICE_OUT) <= 75 and r.TOKENS_IN <= 100_000 # the toy sits inside the pages' ranges (read 2026-10-20)
dearer_again = [t for t in (x24["tokens"] + k * 5e7 for k in range(1, 10_001)) if (c := r.case(round(t), 24))["own"] > c["rent"]] # after the first crossing, in steps of 0.05 billion up to 500 billion past it
ceiling = r.busy_share((now["gpus"] - r.SPARE) * carry, now["gpus"]) # the 11 planned GPUs at their planned load, over the 12 paid for
q5, b14, rate12 = 20 * r.SECONDS_PER_MONTH, r.break_even(14), c12["replies"] / (r.DAYS * 12 * 3600)
g5, carry12 = r.gpus_to_pay_for(q5, 24), r.rates(12)[2]
millionths = lambda: f"{r.TOKENS_IN * r.PRICE_IN:.1f} + {r.TOKENS_OUT * r.PRICE_OUT:.1f}"
wanted = [
f"{B(T)} tokens a month cost {D(now['rent'])} to rent and {D(now['own'])} to run on {now['gpus']} GPUs, because the GPUs are busy {P(now['busy'])} of the time",
f"owning costs {D(c12['own'])} against the same {D(now['rent'])}", f"{B(x24['tokens'])} in the first case",
f"Renting costs {D(now['rent'])}. Owning costs {D(now['own'])}, with {now['gpus']} GPUs busy {P(now['busy'])} of the time. The lines cross at {B(x24['tokens'])} tokens a month.",
f"Owning needs {c4['gpus']} GPUs, costs {D(c4['own'])} and is busy {P(c4['busy'])} of the time",
f"With $0 a month for people the lines cross at {B(people[0]['tokens'])} tokens a month. With $20,000 they cross at {W(people[20_000]['tokens'])} billion.",
f"average {r.TOKENS_IN:,} tokens in and {r.TOKENS_OUT} out. At {B(T)} tokens a month that is {now['replies']:,} replies", f"one reply costs {per_reply * 1e6:.1f} millionths of a dollar",
f"times {r.HOURS_PER_MONTH} hours ({r.DAYS} days of 24)", f"{r.TOKENS_PER_REPLY:,} tokens of one reply ({r.TOKENS_IN:,} in, {r.TOKENS_OUT} out)", f"over {r.DAYS} days of `hours` hours",
f"{B(T)} tokens at {r.TOKENS_PER_REPLY:,} per reply are {now['replies']:,} replies",
f"{r.TOKENS_IN:,} x {r.PRICE_IN} + {r.TOKENS_OUT} x {r.PRICE_OUT} = {millionths()} = {per_reply * 1e6:.1f} millionths of a dollar, which makes {D(now['rent'])} for the month",
f"are {now['replies'] / (r.DAYS * 24 * 3600):.1f} a second. The plan needs {now['gpus'] - r.SPARE} GPUs for that, and the spare makes {now['gpus']}",
f"{now['gpus']} x {r.HOURS_PER_MONTH} x ${r.GPU_HOUR} is {D(now['gpus'] * r.HOURS_PER_MONTH * r.GPU_HOUR)}, and the ${r.PEOPLE:,} for people makes {D(now['own'])}",
f"The {now['gpus']} GPUs serve {now['busy']:.1%} of the replies they could finish. Per million tokens, owning costs ${own_pm:.4f} and renting ${rent_pm:.4f}",
f"Owning first costs no more than renting at {B(x24['tokens'])} tokens a month, with {x24['gpus']} GPUs paid for there. At {B(T)}, owning is {D(now['rent'] - now['own'])} cheaper.",
f"need {c12['gpus']} GPUs, busy {c12['busy']:.1%} of the time", "Each $10,000 more for people, up to $20,000, pushes the crossing out by about 175 billion tokens", "Each $10,000 moves the crossing about 160 to 175 billion tokens.",
f"{rate12:.1f} replies a second", f"so the plan needs {c12['gpus'] - r.SPARE} GPUs and a spare: {c12['gpus']}, against {now['gpus']}. The bill is {D(c12['own'])} against {D(c12['rent'])} for renting, "
f"and the GPUs are busy {c12['busy']:.1%} of the time", f"carries at most the rent of {D(per_reply * carry12)} a month in 12-hour days, and it costs {D(gpu_month)}",
f"busy more than about {math.floor(floor * 100 + 0.5)}% of the month", f"Each ${10_000:,} a month for people moves the crossing out by about 160 to 175 billion tokens. A second spare moves it from {W(x24['tokens'])} to {W(spare[2]['tokens'])} billion.",
f"At $4.00 for a GPU hour the crossing moves from {W(x24['tokens'])} to {W(gpu_hour['4.0']['tokens'])} billion tokens, and at $2.00 to {W(gpu_hour['2.0']['tokens'])} billion.",
f"it is ${round(now['rent_per_million'] * 1000)} for each billion tokens", f"`K` = {round(carry):,} replies a month", f"which is {carry * r.TOKENS_PER_REPLY / 1e9:.1f} billion tokens",
f"`c` = {D(gpu_month)} a month", f"(the ${r.PEOPLE:,} for people plus the spare's {D(gpu_month)})", f"`r` = ${per_reply:.7f} and `c / K` = ${gpu_month / carry:.7f}",
f"`{D(r.PEOPLE + r.SPARE * gpu_month)[1:]} / {per_reply - gpu_month / carry:.7f}` = {smooth / 1e6:.1f} million replies, or {smooth * r.TOKENS_PER_REPLY / 1e9:.0f} billion tokens",
f"The staircase gives {W(x24['tokens'])} billion, within one GPU's carry of {carry * r.TOKENS_PER_REPLY / 1e9:.1f} billion",
f"Here owning costs more than renting again between {W(min(dearer_again))} and {W(max(dearer_again))} billion tokens a month. The code returns the first crossing.",
f"Two lines over 0 to {B(1.25 * T)} tokens a month", f"starts at {D(r.monthly_own(1 + r.SPARE))} and steps up {D(gpu_month)} for each GPU added",
f"They cross at {B(x24['tokens'])} tokens a month, with {x24['gpus']} GPUs paid for there. At {B(T)}, renting costs {D(now['rent'])} and owning {D(now['own'])}.",
f"Rent costs ${rent_pm:.4f} per million tokens at every volume", f"owning is cheaper per million tokens above {P(busy_cross)} busy. At {P(now['busy'])} it is ${own_pm:.4f}.",
f"a planned GPU runs at {P(r.HEADROOM)} of its limit, the spare lowers the top for these {now['gpus']} GPUs to {P(ceiling)}, and a GPU that gets traffic 12 hours of the day is busy at most {P(r.HEADROOM * 12 / 24)}", f"{now['gpus']} GPUs that costs {D(now['own'])} a month",
f"passes under the rent price of ${rent_pm:.4f} at {P(busy_cross)} busy. At {P(now['busy'])} busy, the case of the lesson, owning is ${own_pm:.4f}. The curve ends at {P(ceiling)} busy, the most that {now['gpus'] - r.SPARE} planned GPUs and a spare can reach.",
f"it finishes {flat_out / 1e6:.1f} million replies, worth {D(flat_out * per_reply)} at the rent price", f"{D(gpu_month)} / {D(flat_out * per_reply)} = {floor:.1%} of the month",
f"must exceed {24 * gpu_month / (per_reply * carry):.1f}", f"At 14 hours the crossing is {B(b14['tokens'])} tokens a month, with {b14['gpus']} GPUs. At 13 hours or fewer it does not exist.",
f"is {D(now['rent'])} in every row",
*(f"| {h} | {c['gpus']} | {D(c['own'])} | {P(c['busy'])} | {W(x['tokens']) if x else 'none'} |" for h, (c, x) in by.items()),
*(f"| ${p:,} | {B(x['tokens'])} | {x['gpus']} |" for p, x in people.items()),
"Bars of what owning costs a month for the same " + f"{B(T)} tokens with all the traffic in 24, 18, 12, 8 and 4 hours a day: "
+ ", ".join(f"{by[h][0]['gpus']} GPUs and {D(by[h][0]['own'])}" for h in (24, 18, 12, 8, 4)) + f". A dashed line marks the rent, {D(now['rent'])}.",
"by what the people cost a month: " + "; ".join(f"${p:,}, {B(x['tokens'])}" for p, x in people.items()),
f"none gives {W(spare[0]['tokens'])} billion, one {W(x24['tokens'])} and two {W(spare[2]['tokens'])}.",
f"A GPU hour at $2.00 gives {W(gpu_hour['2.0']['tokens'])} billion tokens and at $4.00 gives {W(gpu_hour['4.0']['tokens'])}.",
f"Rent at $2 and $10 per million tokens gives {B(dear['tokens'])}, so at {B(T)} tokens renting costs {D(r.case(T, 24, price_in=Decimal(2), price_out=Decimal(10))['rent'])} and owning {D(now['own'])}.",
f"at {B(T)} tokens renting costs {D(r.case(T, 24, price_in=Decimal('0.05'), price_out=Decimal('0.25'))['rent'])} and owning {D(now['own'])}.",
f"The toy's ${r.PRICE_IN} and ${r.PRICE_OUT} equal the cheapest row", f"The toy's ${r.GPU_HOUR} sits inside that range",
"from $0.10 to $15 for input and from $0.50 to $75 for output across its listed model rows, retired rows included", "which applies to prompts up to 100,000 tokens", f"the longest prompt in lesson 29's mix is about {round(max(r.cp.chats_of(r.MIX)['prompts']), -3):,} tokens", "Most rows cost far more",
f"at $2 and $10 per million tokens the crossing falls to {B(dear['tokens'])} tokens a month", "a batch discount of 50% on both input and output tokens",
"from $0.69 to $6.99 per GPU per hour, plus sales tax, VAT or GST where it applies", "at $2.79", "its newer models use a tokenizer that produces approximately 30% more tokens for the same text",
"the page lists a batch price of $0.05 and $0.25 for prompts up to 100,000 tokens, and at those prices there is no crossing", # the two price pages, read 2026-10-20
f"{r.TOKENS_IN:,} x {r.PRICE_IN} = {r.TOKENS_IN * r.PRICE_IN:.1f} and {r.TOKENS_OUT} x {r.PRICE_OUT} = {r.TOKENS_OUT * r.PRICE_OUT:.1f} millionths of a dollar, which is "
f"{per_reply * 1e6:.1f} millionths per reply. A million replies cost ${per_reply * 1e6:.2f}.",
f"${r.GPU_HOUR} an hour, so {D(gpu_month)} for a {r.HOURS_PER_MONTH}-hour month", f"It finishes {r.REPLIES_PER_GPU_SECOND:.2f} replies a second at its limit ({flat_out / 1e6:.1f} million in a month)",
f"{D(gpu_month)} / {per_reply:.7f} = {gpu_month / per_reply / 1e6:.1f} million replies. {gpu_month / per_reply / 1e6:.1f} / {flat_out / 1e6:.1f} = {gpu_month / per_reply / flat_out:.1%} of the month",
f"owning cost {D(c12['own'])} and not {D(now['own'])}", f"to {rate12:.1f} replies a second, so the plan needs {c12['gpus'] - r.SPARE} GPUs and a spare: {c12['gpus']}, not {now['gpus']}",
f"{c12['gpus']} x {r.HOURS_PER_MONTH} x ${r.GPU_HOUR} = {D(c12['gpus'] * r.HOURS_PER_MONTH * r.GPU_HOUR)}, plus ${r.PEOPLE:,} for people", f"the rent stays {D(c12['rent'])}",
f"From {W(x24['tokens'])} to {W(people[20_000]['tokens'])} billion tokens a month, with {people[20_000]['gpus']} GPUs paid for there", "about 175 billion tokens for each $10,000",
f"{q5 / r.SECONDS_PER_MONTH:.0f} x {r.SECONDS_PER_MONTH:,} seconds = {q5:,} replies, or {q5 * r.TOKENS_PER_REPLY / 1e9:.1f} billion tokens, below the {W(x24['tokens'])} billion crossing",
f"Renting costs {D(r.monthly_rent(q5))}. Owning takes {g5} GPUs: {g5} x {r.HOURS_PER_MONTH} x ${r.GPU_HOUR} = {D(g5 * r.HOURS_PER_MONTH * r.GPU_HOUR)}, plus ${r.PEOPLE:,} for people = {D(r.monthly_own(g5))}",
]
visible = {
"cost-per-million.svg": [f"rent ${rent_pm:.4f}", f"own, {now['gpus']} GPUs", f"{P(busy_cross)} busy", f"${own_pm:.4f}", f"for a fixed set of {now['gpus']} GPUs that costs {D(now['own'])} a month", f"at most {P(ceiling)}",
f"The curve ends at {P(ceiling)} busy, the most that {now['gpus'] - r.SPARE} planned GPUs and a spare can reach."],
"people-cost.svg": [f"people {D(p)} a month: {x['gpus']} GPUs" for p, x in people.items()] + [f">{B(x['tokens'])}<" for x in people.values()],
"rent-or-own.svg": [f">{B(x24['tokens'])}<", f"rises ${round(now['rent_per_million'] * 1000)} for each billion tokens", f"Owning starts at {D(r.monthly_own(1 + r.SPARE))}", f"steps up {D(gpu_month)} for each GPU added",
f"with {x24['gpus']} GPUs paid for there. At {B(T)}, renting costs {D(now['rent'])} and owning {D(now['own'])}."],
"same-tokens-fewer-hours.svg": [f"{h} hours a day: {by[h][0]['gpus']} GPUs, {P(by[h][0]['busy'])} busy" for h in (24, 18, 12, 8, 4)]
+ [f"{D(by[h][0]['own'])}, {'cheaper' if by[h][0]['own'] < by[h][0]['rent'] else 'dearer'}" for h in (24, 18, 12, 8, 4)] + [f"renting, {D(now['rent'])} a month"],
}
missing = [f"lesson.md: {fragment!r}" for fragment in wanted if fragment not in prose]
missing += [f"{name}: {fragment!r}" for name, fragments in visible.items() for fragment in fragments if fragment not in figures[name]]
assert not missing, "not found: " + "; ".join(missing)
for name, text in figures.items(): # one palette, a white page, a title and a description, no script and no address
assert not re.search(r"0072B2|56B4E9|D55E00", text, re.I) and "<title>" in text and "<desc>" in text and 'fill="#ffffff"' in text, name
assert "<script" not in text and not WEB_ADDRESS.search(text.replace("ht" + "tp://www.w3.org/2000/svg", "")), name
for path in CODE.glob("*.py"):
assert not WEB_ADDRESS.search(path.read_text(encoding="utf-8")), path.nameShow code
"""Guards for ../widget.html: its data block, that its sums give the same numbers, words and chart paths as rent_vs_own.py, and what a reader sees and can press."""
import json
import random
import re
import subprocess
import rent_vs_own as r
from embed_widget import CONFIG_BLOCK, WIDGET, config, embed
HTML = WIDGET.read_text(encoding="utf-8")
BASE = (300, 24, 10_000, 0.10, 0.50, 3.0, 1)
HOURS_SHOWN = (1, 4, 8, 12, 13, 14, 18, 24)
def grid():
"""Every monthly tokens setting against the hours that flip the answer and three people costs; every value of every other control one at a time, at three sizes; and a
seeded sample of all seven at once."""
cases = [(t, h, p, *BASE[3:]) for t in r.TOKENS_B for h in HOURS_SHOWN for p in (0, 10_000, 40_000)]
values = {1: range(r.HOURS_RANGE[0], r.HOURS_RANGE[1] + 1), 2: range(r.PEOPLE_RANGE[0], r.PEOPLE_RANGE[1] + 1, r.PEOPLE_RANGE[2]), 3: r.PRICES_IN, 4: r.PRICES_OUT,
5: r.GPU_HOURS, 6: r.SPARES}
for t in (1, 300, 2000):
for slot, options in values.items():
cases += [(t, *BASE[1:slot], v, *BASE[slot + 1:]) for v in options]
pick = random.Random(30)
cases += [(pick.choice(r.TOKENS_B), pick.randint(1, 24), pick.randrange(0, 40_001, 1000), pick.choice(r.PRICES_IN), pick.choice(r.PRICES_OUT), pick.choice(r.GPU_HOURS),
pick.choice(r.SPARES)) for _ in range(300)]
return cases
# Reads the sums out of the widget between its marker comments, runs every setting of the grid with Node and compares the three lines of words and the chart paths with the
# ones Python wrote into the fixture, character for character.
PARITY = r"""
import fs from 'node:fs';
const [fixture, widget] = process.argv.slice(2);
const fx = JSON.parse(fs.readFileSync(fixture, 'utf8'));
const html = fs.readFileSync(widget, 'utf8');
const cfg = JSON.parse(html.match(/<script type="application\/json" id="cp-config">([\s\S]*?)<\/script>/)[1]);
const js = html.slice(html.indexOf('/* ---------- the model ('), html.indexOf('/* ---------- display'));
const api = new Function('cfg', "'use strict';" + js + '; return {describe};')(cfg);
const bad = [];
fx.cases.forEach((c) => {
const got = api.describe(...c.args);
if (JSON.stringify(got) !== JSON.stringify(c.want)) bad.push([c.args, JSON.stringify(got).slice(0, 400), JSON.stringify(c.want).slice(0, 400)]);
});
console.log(JSON.stringify({ checks: fx.cases.length, bad: bad.slice(0, 3) }));
process.exit(bad.length ? 1 : 0);
"""
# Runs the page script against a small stand-in for the DOM, moves the sliders and reads back what a reader sees and hears.
PAGE = r"""
import fs from 'node:fs';
const [widget, fixture] = process.argv.slice(2);
const html = fs.readFileSync(widget, 'utf8');
const fx = JSON.parse(fs.readFileSync(fixture, 'utf8'));
const config = html.match(/<script type="application\/json" id="cp-config">([\s\S]*?)<\/script>/)[1];
const code = html.match(/<script>([\s\S]*?)<\/script>/)[1];
function el(tag) {
const e = { tag, children: [], listeners: {}, value: '', checked: false, style: {}, attrs: {}, textContent: '' };
e.setAttribute = (k, v) => { e.attrs[k] = v; };
e.getAttribute = (k) => e.attrs[k];
e.appendChild = (c) => { e.children.push(c); };
e.addEventListener = (type, f) => (e.listeners[type] = e.listeners[type] || []).push(f);
return e;
}
const page = { root: el('html'), messages: [], parent: { other: true } };
[...html.matchAll(/id="(cp-[^"]+)"/g)].forEach((m) => { page[m[1]] = el('x'); });
page['cp-config'].textContent = config;
const doc = { getElementById: (id) => page[id], documentElement: page.root, createElement: el };
const win = { addEventListener: (type, f) => page.messages.push(f) };
new Function('document', 'window', 'parent', code)(doc, win, page.parent);
const fire = (e, type) => (e.listeners[type] || []).forEach((f) => f({ target: e }));
const bad = [];
const want = (what, got, expected) => { if (JSON.stringify(got) !== JSON.stringify(expected)) bad.push([what, JSON.stringify(got).slice(0, 400), JSON.stringify(expected).slice(0, 400)]); };
const cfg = JSON.parse(config);
const shown = () => ({ headline: page['cp-headline'].textContent, verdict: page['cp-verdict'].textContent, detail: page['cp-detail'].textContent,
caption: page['cp-caption'].textContent, rent: page['cp-rent'].attrs.d, own: page['cp-own'].attrs.d, cross: page['cp-cross'].attrs.d, mark: page['cp-mark'].attrs.d,
dots: [page['cp-dot-rent'].attrs.cx, page['cp-dot-rent'].attrs.cy, page['cp-dot-own'].attrs.cx, page['cp-dot-own'].attrs.cy],
outs: ['tokens', 'hours', 'people', 'pin', 'pout', 'gpu', 'spare'].map((k) => String(page['cp-' + k + '-out'].textContent)) });
const set = (a) => {
page['cp-tokens'].value = String(cfg.tokensB.indexOf(a[0])); page['cp-hours'].value = String(a[1]); page['cp-people'].value = String(a[2]);
page['cp-pin'].value = String(cfg.pricesIn.indexOf(a[3])); page['cp-pout'].value = String(cfg.pricesOut.indexOf(a[4]));
page['cp-gpu'].value = String(cfg.gpuHours.indexOf(a[5])); page['cp-spare'].value = String(cfg.spares.indexOf(a[6]));
};
want('opening controls', ['tokens', 'hours', 'people', 'pin', 'pout', 'gpu', 'spare'].map((k) => page['cp-' + k].value),
[String(cfg.tokensB.indexOf(cfg.opening.tokens_b)), String(cfg.opening.hours), String(cfg.opening.people), String(cfg.pricesIn.indexOf(cfg.opening.price_in)),
String(cfg.pricesOut.indexOf(cfg.opening.price_out)), String(cfg.gpuHours.indexOf(cfg.opening.gpu_hour)), String(cfg.spares.indexOf(cfg.opening.spare))]);
want('opening state', shown(), fx.opening);
want('nothing spoken yet', page['cp-live'].textContent, '');
set(fx.peaky.args);
fire(page['cp-hours'], 'input');
want('4 hours, dragging', shown(), fx.peaky.want);
want('dragging stays silent', page['cp-live'].textContent, '');
fire(page['cp-hours'], 'change');
want('release speaks', page['cp-live'].textContent, [fx.peaky.want.headline, fx.peaky.want.verdict, fx.peaky.want.detail].join(' '));
want('words on the controls', [page['cp-hours'].attrs['aria-valuetext'], page['cp-tokens'].attrs['aria-valuetext'], page['cp-people'].attrs['aria-valuetext'],
page['cp-chart'].attrs['aria-label'].endsWith(fx.peaky.want.headline)], ['4 hours a day', '300 billion tokens a month', '$10,000 a month', true]);
fx.others.forEach((c) => { set(c.args); fire(page['cp-gpu'], 'input'); want('state ' + c.args.join('/'), shown(), c.want); });
const theme = (source, name) => page.messages.forEach((f) => f({ data: { type: 'spicyshark:theme', theme: name }, source }));
theme({ other: 'frame' }, 'dark');
want('theme from another sender is ignored', page.root.attrs['data-theme'], undefined);
theme(page.parent, 'dark');
want('theme from the host page', page.root.attrs['data-theme'], 'dark');
console.log(JSON.stringify({ bad: bad.slice(0, 3) }));
process.exit(bad.length ? 1 : 0);
"""
def run_node(tmp_path, script, *args):
path = tmp_path / "check.mjs"
path.write_text(script, encoding="utf-8")
done = subprocess.run(["node", str(path), *args], capture_output=True, text=True, timeout=120)
assert done.returncode == 0, done.stdout + done.stderr
return json.loads(done.stdout)
def seen(*args):
"""What the page shows for one setting, in the form the PAGE script reads back."""
d = r.describe(*args)
c = d["chart"]
tokens_b, hours, people, price_in, price_out, gpu_hour, spare = args
line = lambda x: f"M{x},{r.PAD} V{r.H - r.PAD}"
return {"headline": d["headline"], "verdict": d["verdict"], "detail": d["detail"], "caption": c["caption"], "rent": c["rent"], "own": c["own"],
"cross": "" if c["cross"] is None else line(c["cross"]), "mark": line(c["mark"]), "dots": [c["mark"], c["dots"][0], c["mark"], c["dots"][1]],
"outs": [f"{tokens_b:,} billion", str(hours), f"${people:,}", f"${price_in:.2f}", f"${price_out:.2f}", f"${gpu_hour:.2f}", str(spare)]}
def test_widget_data_block_matches_the_python_module():
assert embed(HTML) == HTML
block = json.loads(CONFIG_BLOCK.search(HTML).group(2))
assert block == json.loads(json.dumps(config()))
assert block["opening"] == {"tokens_b": 300, "hours": 24, "people": 10_000, "price_in": 0.1, "price_out": 0.5, "gpu_hour": 3.0, "spare": 1}
for name, (low, high, step) in {"hours": r.HOURS_RANGE, "people": r.PEOPLE_RANGE}.items(): # the sliders cover the settings the grid covers
assert re.search(r'id="cp-%s" min="%d" max="%d" step="%d"' % (name, low, high, step), HTML), name
def test_widget_gives_the_same_numbers_words_and_paths_as_python(tmp_path):
cases = [{"args": list(g), "want": r.describe(*g)} for g in grid()]
words = " ".join(c["want"]["headline"] + c["want"]["verdict"] + c["want"]["detail"] for c in cases)
assert "never catches up" in words and "Renting costs less" in words and "Owning costs less" in words and "no spare" in words and "2 spares" in words # the grid reaches both answers
assert any(c["want"]["chart"]["cross"] is None for c in cases) and any(c["want"]["chart"]["cross"] is not None for c in cases)
(tmp_path / "fx.json").write_text(json.dumps({"cases": cases}), encoding="utf-8")
assert run_node(tmp_path, PARITY, str(tmp_path / "fx.json"), str(WIDGET))["checks"] == len(cases)
def test_widget_page_shows_the_opening_state_and_follows_the_controls(tmp_path):
opening, peaky = seen(*BASE), (300, 4, 10_000, 0.10, 0.50, 3.0, 1)
assert opening["headline"] == "Below about 225 billion tokens a month, renting costs less."
assert opening["verdict"] == "At 300 billion tokens a month, renting costs $39,240 and owning $35,920. Owning costs less."
assert seen(*peaky)["headline"] == "At these settings owning never catches up with renting." and seen(*peaky)["cross"] == ""
others = [(5, 18, 0, 2.0, 10.0, 1.0, 0), (2000, 13, 40_000, 10.0, 50.0, 8.0, 2), (1, 1, 0, 0.5, 2.0, 1.5, 0), (750, 14, 20_000, 4.0, 20.0, 6.0, 1)]
fixture = {"opening": opening, "peaky": {"args": list(peaky), "want": seen(*peaky)}, "others": [{"args": list(o), "want": seen(*o)} for o in others]}
(tmp_path / "fx.json").write_text(json.dumps(fixture), encoding="utf-8")
run_node(tmp_path, PAGE, str(WIDGET), str(tmp_path / "fx.json"))
def test_widget_is_small_themed_and_uses_only_the_site_palette():
body = HTML[HTML.index('<div class="cp-root">'):HTML.index("<script")]
visible = re.sub(r"<details.*?</details>", "", body, flags=re.S)
assert len(re.findall(r"<input|<select|<button", visible)) <= 5 and "More settings" in body and "A toy: round prices" in visible # at most five controls on show, and the caveat is always on show
assert len(HTML.splitlines()) < 300 and "light-dark(#ffffff, #0a0f17)" in HTML and "spicyshark:theme" in HTML and "prefers-reduced-motion" in HTML
assert not re.search(r"0072B2|56B4E9|D55E00|ht" r"tps?://|<img|@import", HTML, re.I)