---
title: "Why is my AI app bill higher than I expected?"
description: "AI services bill every token sent and every token written back, and each chat turn sends the whole chat again, so a bill grows faster than the chat does."
topic: "AI systems"
chapter: "use-an-ai-service"
order: 7
minutes: { plain: 5, deep: 12 }
prerequisites: ["Reading short Python functions", "What a token is"]
seo_title: "Why is my AI API bill so high?"
date: 2026-10-19
---
# Why is my AI app bill higher than I expected?
You add up what you type and what the bot writes back, multiply by the price, and expect a small bill. Take a chat of ten turns (a turn is one message from you and its reply). Each message is 60 tokens (a token is a chunk of text, often a short word or part of one), each reply is 200 tokens, and the app sends a 500-token system prompt, the standing instructions, with every message. At example prices of $2 per million tokens sent and $10 per million written, what you typed and what came back cost $0.0212. Billed the way providers bill it, the same chat costs $0.0546, 2.6 times as much, because every message sends the whole chat again.
## In one minute
Your AI bill is higher than you expect because the service charges for every token it reads and writes, and each new message makes it read the whole chat again. Picture a consultant who bills by the page and re-reads the whole file before each question: even a one-line tenth question carries every page before it. In the price rows we read, writing costs more than reading, and some models also write long hidden working-out that is billed as writing. The fixes all come down to sending less on each turn and watching the numbers.
## Explore the numbers
[Open the widget](https://spicyshark.co/learn/why-ai-bill-is-high/widget.html). Set Turns per chat to 30 and watch each turn cost more than the last as the orange re-sent history grows. Then tick "Keep last 3 turns" and watch the monthly total fall. Prices, message sizes and the retry rate sit under More settings.
The starting prices, $2 and $10 per million tokens, are example figures, not a quote; each provider publishes its own: [Anthropic](https://platform.claude.com/docs/en/about-claude/pricing), [OpenAI](https://developers.openai.com/api/docs/pricing) and [Google](https://ai.google.dev/gemini-api/docs/pricing) (read 2026-10-19). Google's [token guide](https://ai.google.dev/gemini-api/docs/tokens) says 100 tokens are about 60 to 80 English words (read 2026-10-19), which is how the widget turns tokens into words. The starting chat costs $0.0546, and 100 such chats a day for 30 days cost $163.80.
## How it works
### What the bill counts
An AI service charges by the token and keeps two counts. **Input tokens** are everything your app sends in one call (one request to the service): your new message and everything that travels with it. **Output tokens** are everything the model writes back. Each has its own price per million tokens, and writing costs more: Anthropic's pricing page lists one model at $2 per million input tokens and $10 per million output tokens, and OpenAI's page lists a model at the same two prices (both read on 2026-10-19), five times as much for output. More in [What is a token?](https://spicyshark.co/learn/what-is-a-token/).
### Every turn sends the whole chat again
A model keeps nothing between calls, so to answer your tenth message the app sends all ten messages and nine replies. OpenAI's conversation-state guide says that even when you chain responses, all earlier input tokens in the chain are billed as input again (read 2026-10-19).
The new message is 60 tokens, yet turn 4 carries 1,340 input tokens and turn 10 carries 2,900. Of the 17,300 input tokens in the ten-turn chat, 11,700 (67.6%) are earlier chat sent again, 5,000 (28.9%) the system prompt and only 600 (3.5%) your new messages. Anthropic's pricing page says the same system prompt, document or conversation history is processed again on each request (read 2026-10-19).
Ten turns cost $0.0546. Thirty turns cost $0.3198, nearly six times as much for three times the length.
### What else rides along
Whatever your app attaches to every call is input every time: the system prompt, and any document you paste in. A 2,000-token system prompt in place of the 500-token one lifts the ten-turn chat from $0.0546 to $0.0846. Retries add more: a failed call sent again carries the same input a second time, and if one call in ten is sent twice the ten-turn chat costs $0.06006. The error pages read for this lesson do not say whether a failed call is billed.
### Hidden reasoning is billed as output
Some models, called reasoning models, work out an answer in writing before they give it. You may never see that text, yet these **reasoning tokens** are billed as output. OpenAI's reasoning guide says so, Anthropic's page on steering thinking says the same of its thinking tokens, and Google's pricing page labels its output price "including thinking tokens" (all read 2026-10-19). OpenAI's guide adds that one reply can use from a few hundred to tens of thousands of them.
With 800 hidden tokens behind each 200-token reply, the example chat costs $0.1346 instead of $0.0546.
### Cutting the history
Send only the last few turns and the cost stops growing with the chat: keep the last 3 earlier turns and every call from turn 4 on carries 1,340 input tokens. The ten-turn chat then costs 20.0% less, the thirty-turn chat 57.1% less. The model no longer sees what you cut, which [Why does a chatbot forget?](https://spicyshark.co/learn/why-chatbots-forget/) explains.
## What this means when you build
- **Estimate with the whole chat, not one message.** Add the system prompt and every earlier turn to each call: counting only what you type and what comes back was 2.6 times too low for ten turns and 5.0 times too low for thirty. Log the token counts each response reports.
- **Send less.** Keep the system prompt short, trim or summarize the history, and cap the length of replies: replies capped at 100 tokens bring the ten-turn chat from $0.0546 to $0.0356. On a reasoning model leave room in the cap: OpenAI's guide warns you can pay for input and reasoning and get no visible reply if the cap runs out first.
- **Use a smaller model for easy steps.** Anthropic's page lists $0.10 and $0.50 per million tokens for its small model, Haiku 5.5, against $2 and $10 for Sonnet 5.5, for prompts up to 100,000 tokens (read 2026-10-19): 20 times less for the example chat.
- **Reuse a repeated start.** Providers can store the start of a prompt that repeats and bill reading it back at a lower price: $0.10 per million tokens against $2 for ordinary input, on one Anthropic row and one OpenAI row (read 2026-10-19). [What is prompt caching?](https://spicyshark.co/learn/what-is-prompt-caching/) explains when it pays; the linked pricing pages list each cached-input price.
- **Set a spending limit or alert** wherever your provider offers one, and keep the key on your own server: [An API key in the browser](https://spicyshark.co/learn/api-key-in-the-browser/) shows how.
## Go deeper
*Optional. This part is for readers who write Python. The code is in the `code/` folder next to this lesson, and `python3 show_numbers.py` prints the numbers below that the code computes. Sizes are in tokens, and prices are dollars per million tokens.*
### The naive estimate, and why it fails
The first estimate most people write prices what they type and what comes back, once for each turn:
```python
def naive_bill(chat, price_in, price_out):
tokens_in = chat.turns * chat.message
tokens_out = chat.turns * (chat.reply + chat.reasoning)
cost_in = tokens_in * _price(price_in, "price_in")
cost_out = tokens_out * _price(price_out, "price_out")
return (cost_in + cost_out) / PER
```
(The docstring is left out here. `_price` turns a price into a `Decimal`, so cents are exact, and refuses a negative one.) It gives $0.0212 for the ten-turn example and $0.0636 for thirty turns. The real bills are $0.0546 and $0.3198, which are 2.6 and 5.0 times higher. The estimate leaves out two things that every call carries, the system prompt and the earlier turns, and it grows in a straight line with the number of turns while the real bill does not.
### One turn by hand
Turn `k` sends `k - 1` earlier turns, and each earlier turn is a message and its reply: 60 + 200 = 260 tokens. So the input of turn `k` is the system prompt (500 tokens), plus 260 x (k - 1), plus the new message (60 tokens). The output is the 200-token reply every turn.
| turn | input | cost |
|---|---|---|
| 1 | 560 | $0.00312 |
| 2 | 820 | $0.00364 |
| 3 | 1,080 | $0.00416 |
| 4 | 1,340 | $0.00468 |
| 10 | 2,900 | $0.0078 |
The cost of turn 4 by hand is 1,340 x 2 + 200 x 10 = 2,680 + 2,000 = 4,680 millionths of a dollar, which is $0.00468. All ten turns together send 17,300 input tokens and write 2,000, so the bill is $0.0346 for input plus $0.02 for output, $0.0546 in all. Output is 36.6% of the money while it is only 10.4% of the tokens.
### Adding it up without a loop
The brute-force way walks through the turns the way a chat app does. `history` holds the sizes of the earlier messages, and trimming keeps the last `2 * keep` of them:
```python
def simulate(chat):
history = [] # earlier message sizes, oldest first
total_in = total_out = 0
for _ in range(chat.turns):
kept = history
if chat.keep is not None:
kept = history[max(0, len(history) - 2 * chat.keep):]
total_in += chat.system + sum(kept) + chat.message
total_out += chat.reply + chat.reasoning
history += [chat.message, chat.reply]
return total_in, total_out
```
The loop is the definition, and the tests use it as the reference. A closed form is shorter to compute and shows the shape. Turn `k` re-sends `k - 1` earlier turns, and 0 + 1 + 2 + ... + (n - 1) = n(n - 1)/2, which is 45 for n = 10. Every turn also sends the system prompt and the new message, which is the `fixed` line below:
```python
def closed_form(chat):
n = chat.turns
t = n - 1 if chat.keep is None else min(n - 1, chat.keep)
earlier_turns = t * (t + 1) // 2
if chat.keep is not None:
earlier_turns += chat.keep * (n - 1 - t)
fixed = n * (chat.system + chat.message)
total_in = fixed + (chat.message + chat.reply) * earlier_turns
return total_in, n * (chat.reply + chat.reasoning)
```
With nothing trimmed, `t` is `n - 1`, the `if` adds nothing and `earlier_turns` is (n - 1)n/2. By hand: 10 x 560 + 260 x 45 = 5,600 + 11,700 = 17,300, the same as the loop. The n(n - 1)/2 term makes the bill grow with the square of the chat length, so a chat twice as long needs about four times the history. The longer chats show it, first in tokens:
| turns | input | output |
|---|---|---|
| 10 | 17,300 | 2,000 |
| 20 | 60,600 | 4,000 |
| 30 | 129,900 | 6,000 |
Then in dollars, next to the cost if every turn were a fresh start:
| turns | cost | fresh start |
|---|---|---|
| 10 | $0.0546 | $0.0312 |
| 20 | $0.1612 | $0.0624 |
| 30 | $0.3198 | $0.0936 |
A fresh start is a one-turn chat with the same system prompt, which costs $0.00312. Doubling the chat from 10 to 20 turns takes the input from 17,300 to 60,600 tokens, 3.5 times as many.
### Trimming turns the curve into a line
With `keep` set to `N`, the first `N + 1` turns re-send 0, 1, ..., `N` earlier turns, and every later turn re-sends `N`. That is N(N + 1)/2 + N x (n - 1 - N) earlier turns in all, the two lines that build `earlier_turns` above. For N = 3 and n = 10 it is 6 + 18 = 24, so the input is 5,600 + 260 x 24 = 11,840 tokens against 17,300. From turn 4 on, every turn sends 1,340 input tokens, so the total grows in a straight line: 38,640 tokens for 30 turns against 129,900 untrimmed. Across a month of 100 chats a day (3,000 chats at 30 days), the totals are, sending all turns or only the last 3 earlier turns:
| chat | all turns | last 3 |
|---|---|---|
| 10 turns | $163.80 | $131.04 |
| 30 turns | $959.40 | $411.84 |
### Reasoning and retries
The `reasoning` field of `Chat` adds `reasoning` tokens to the output of every turn, and to nothing else. With 800 hidden tokens per reply, the output goes from 2,000 to 10,000 tokens and the bill rises by $0.08, from $0.0546 to $0.1346. Retries live in `bill`, which takes the share of calls made twice as `retry_rate`:
```python
tokens_in, tokens_out = closed_form(chat)
cost_in = tokens_in * price_in / PER
cost_out = tokens_out * price_out / PER
retry_cost = (cost_in + cost_out) * retry_rate
return Bill(tokens_in, tokens_out, cost_in, cost_out, retry_cost)
```
(These are the last lines of `bill`.) The cost model counts a retry as one more full call with the same input, so one call in ten adds a tenth of the bill: $0.00546 on $0.0546, $0.06006 in all.
### What the cost model leaves out
In this lesson "the cost model" is the arithmetic in `bill.py`, and "the model" alone is the AI model.
- **Reasoning may be sent back.** The cost model treats reasoning tokens as output only and leaves them out of the history. Providers differ. Anthropic's page on steering thinking (read 2026-10-19) says that thinking blocks from earlier turns that stay in the context are billed as input, on every earlier turn for some models and on the last one for the rest. OpenAI's guide on conversation state (read 2026-10-19) says that models with persisted reasoning can use `reasoning.context: "all_turns"` to bring reasoning from earlier turns into the next request, and it does not say how that is billed. If yours does this, the history part is larger than the cost model says.
- **A failed call may not cost a full call.** The cost model bills a failed call in full, which is an upper bound. Anthropic's errors page says its SDK retries transient failures twice by default, with exponential backoff, and OpenAI's error-codes guide advises backoff with jitter and a limit on retries (both read 2026-10-19). Neither page says whether a failed request is billed.
- **Prices that change.** The real prices below move with prompt length for some models, and the cost model uses one flat price. It also leaves out cached input, which costs less, and the 50% discount Anthropic's page lists for its Batch API.
- **Token counts depend on the tokenizer.** Anthropic's token-counting page says its newer tokenizers give about 30% more tokens for the same text and that billing follows, and that the count it returns is an estimate. Count with your provider's own tool.
Sonnet 5.5 and Haiku 5.5 are Anthropic's, gpt-6.1-sol and gpt-6-luna are OpenAI's, and Gemini 3.1 Pro, here the Preview release, is Google's. Haiku's prices are for prompts up to 100,000 tokens, the OpenAI rows are the standard tier at short context, and Gemini's are for prompts up to 200,000 tokens.
| model | input | output |
|---|---|---|
| Sonnet 5.5 | $2 | $10 |
| Haiku 5.5 | $0.10 | $0.50 |
| gpt-6.1-sol | $2.00 | $10.00 |
| gpt-6-luna | $0.10 | $0.50 |
| Gemini 3.1 Pro | $2.00 | $12.00 |
These are dollars per million tokens, read on 2026-10-19. Output costs five times as much as input in four rows and six times in the Gemini row. The example chat costs $0.0546 at $2 and $10 and $0.00273 at $0.10 and $0.50, which is 20 times less.
### Proof it works
```bash
cd lessons/why-ai-bill-is-high/code && python3 -m pytest -q
python3 embed_widget.py --check
```
The tests check these claims. The closed form equals the simulation on 960 different chats, trimmed and untrimmed, with and without reasoning tokens, and the per-turn parts add up to the same totals. The example chat gives the hand-computed turns, 17,300 input and 2,000 output tokens and $0.0546. Untrimmed, each turn adds 260 tokens more to the total than the turn before, which is what makes it quadratic. Trimmed, once the window is full every turn adds the same amount, and trimming never costs more. Reasoning adds output cost only. The retry term equals the expected cost over every choice of which of six turns are retried, worked out with exact fractions, and the naive estimate misses exactly the system prompt and the history. The answers in Check yourself are pinned, and nonsense sizes and negative prices are refused. Six more tests guard the widget's starting numbers, its markup and its cost arithmetic; none of them checks how the widget draws or words its results. For the arithmetic, one test runs the widget's JavaScript under Node.js (so it needs Node.js installed) on 3,783 combinations of chat, prices and retry rate and requires the same tokens and dollars as the Python. Changing the trimmed sum, dropping the reasoning term or the retry term, or changing the trimmed tail makes a test fail. The embed check fails if the defaults in `widget.html` drift from `bill.py`.
### Where this is used in practice
- **Usage fields.** OpenAI returns `output_tokens_details.reasoning_tokens`, Anthropic returns `usage.output_tokens_details.thinking_tokens` with `output_tokens` as the total you are billed for, and Google's usage object has `total_input_tokens`, `total_output_tokens` and `total_thought_tokens`. Log them per call.
- **Caps.** OpenAI's `max_output_tokens` and Anthropic's `max_tokens` cap the reasoning and the visible reply together.
- **Counting before you send.** Google has `count_tokens`. Anthropic's counting endpoint is free but rate-limited.
- **Chained conversations.** OpenAI's `previous_response_id` saves you from re-sending the history yourself, but not from paying for it.
## Skills this lesson builds
- Estimate the bill of a chat feature from message sizes and prices, with the history term included.
- Write a cost model and check its closed form against a turn-by-turn simulation.
- Read a provider's usage fields and tell input, output and reasoning tokens apart.
- Compare fixes (trim, cap, shorten, smaller model, cached start) by the money each saves.
- Name what a cost model leaves out, such as price tiers, discounts and billed failures.
**Portfolio project:** write a small command-line tool that reads a log of calls (input and output token counts per call), prints the cost per chat and per turn and the share spent on earlier turns sent again, and shows the saving from trimming to the last N turns. Publish a chart of cost per turn for a chat you ran yourself.
## Check yourself
1. A ten-turn chat has 60-token messages, 200-token replies and a 500-token system prompt. Your sum of what you type and what comes back says $0.0212, and the bill says $0.0546. Where did the rest come from?
Answer
The sum counted each message once. The app sends the system prompt and all the earlier turns again with every message, so the input is 17,300 tokens, not 600. That is $0.0346 for input plus $0.02 for output.Answer
Each turn sends every earlier turn again, so each call carries 260 more input tokens than the call before: turn 10 sends 2,900 and turn 30 sends 8,100. Total input is 129,900 tokens against 17,300, and the cost is $0.3198 against $0.0546.Answer
Untrimmed: 4 x 110 + 30 x (0 + 1 + 2 + 3) = 440 + 180 = 620 tokens. Keeping 1 earlier turn: 4 x 110 + 30 x (0 + 1 + 1 + 1) = 530 tokens. Output is 4 x 20 = 80 tokens. The first case costs 620 x 2 + 80 x 10 = 2,040 millionths of a dollar, $0.00204.Answer
Reasoning tokens are billed as output, so it is 750 x 10 = 7,500 millionths of a dollar, $0.0075. Without the reasoning it would be 150 x 10 = $0.0015, five times less.Answer
Each earlier turn is 100 tokens. Everything: 8 x 230 + 100 x 28 = 1,840 + 2,800 = 4,640 tokens. Trimmed, the turns re-send 0, 1, 2, 2, 2, 2, 2, 2 earlier turns, 13 in all: 1,840 + 1,300 = 3,140 tokens, which is 32.3% fewer.