---
title: "Why can't AI count the letters in a word?"
description: "An AI model reads chunks of text as numbers, not letters, so a letter can hide inside a chunk. Spelling the word out, or running code, gets around it."
topic: "AI systems"
chapter: "how-ai-answers"
order: 2
minutes: { plain: 5, deep: 12 }
prerequisites: ["Reading short Python functions", "What a token is"]
seo_title: "Why can't AI count the r's in strawberry?"
date: 2026-10-19
---
# Why can't AI count the letters in a word?
Ask a person how many r's are in "strawberry" and they look at the letters and say 3. A typical chatbot is given chunk numbers, not those letters. In the small toy version built for this lesson, the word is cut into 7 chunks, and 2 of the 3 r's sit inside chunks that hold other letters too.
## In one minute
An AI model has trouble counting the letters in a word because it usually does not read letters. A program called a tokenizer first cuts your text into chunks called tokens, and the model is given one number for each chunk. Picture being handed numbered cards instead of a word, with the letters printed on the back of each card, where you cannot see them. To count the r's you would have to remember what is on the back of every card. A model learns many of those spellings from its training text, but it has to recall them rather than look. Writing the word out letter by letter first puts one letter in each chunk, and a tool that runs code can count exactly.
## Explore the numbers
[Open the widget](https://spicyshark.co/learn/why-ai-cant-count-letters/widget.html). It starts on strawberry and the letter r. The top row shows the letters you see. The row beneath shows the chunks the tokenizer cuts, and the row under that shows how many r's are inside each chunk. Orange marks the letter you chose. Tick "Spell it out" and watch every letter become a chunk of its own, then try words of your own. The widget uses this lesson's toy tokenizer, built the way [What is a token?](https://spicyshark.co/learn/what-is-a-token/) builds one, and it runs in your browser with no network. Real tokenizers cut words in other places, as the list below shows.
## How it works
### What the model is given
Each chunk the tokenizer cuts is a token, and each token has a **token id**, a plain number such as 263. The model is given the ids, not the text. In the toy, strawberry has 10 letters and becomes 7 chunks: st, r, a, w, b, er and ry, with the ids 292, 114, 97, 119, 98, 263 and 587.
Count the r's by looking for the id of a lone r, 114, and you find one. The real answer is 3, because two of the r's are inside er and ry.
### The number does not spell its chunk
An id is a label. The tokenizer keeps a table that says 263 stands for er, but the model is given only the 263, and nothing in that number holds an e or an r.
What each id stands for is learned in training. A 2024 test of character skills, CUTE, found that most models it tried seem to know how their tokens are spelled, yet do not use that well to change a text (read 2026-10-19). A second 2024 paper found that most models it tested could not reliably do simple letter tasks that people find easy (read 2026-10-19). Both studied the models of their time.
### Real tokenizers cut elsewhere
Real tokenizers have far bigger tables than the toy, about 100,000 and 200,000 ids for the two below, called cl100k_base and o200k_base, which come with tiktoken, a free code library from OpenAI. These lines show where each one cuts a word, run with tiktoken 0.14.0 on 2026-10-19. A dot marks each cut.
- **strawberry:** str · aw · berry in cl100k_base; st · raw · berry in o200k_base.
- **Strawberry:** Str · aw · berry in both.
- **STRAWBERRY:** ST · RAW · B · ERRY in both.
- **banana:** one chunk in both.
- **Mississippi:** Miss · issippi in both.
- **tokenizer:** one chunk in cl100k_base; token · izer in o200k_base.
- **unbelievable:** un · belie · vable in cl100k_base; un · bel · ievable in o200k_base.
Banana is one chunk in both, so its three a's are inside a single id. Capitals change the cut, and so does a space before the word.
### Why spelling it out helps
Write the word as s-t-r-a-w-b-e-r-r-y and both tokenizers cut it between the letters. Each chunk then holds one letter, most with a hyphen, and the three r chunks have the same id: 6335 in o200k_base.
Now counting r's means counting repeats of one id. A model that writes a word out this way as its first step gives itself that spelled-out text to read. How well that works depends on the model, so test yours.
A tool that runs code avoids the problem. The code is given the text as letters, so the count is exact for the text the code is given. Providers document such tools: OpenAI's Code Interpreter and Anthropic's code execution tool both let a model write and run Python code in a sandbox, a sealed space that cannot reach your other files (read 2026-10-19).
## What this means when you build
- **Use code for exact letter work.** Counting letters, checking length, reversing text and finding the third character are exact jobs on a string of letters. Do them in your own code, not in the model's head.
- **Give the model a code tool when users ask for such answers.** Both providers named above document one.
- **Otherwise ask for the word spelled out first.** Then test it on your model and your words, because behaviour changes between versions and one correct answer proves little.
- **Test variants.** Capitals, a space before a word and hyphens change the chunks, so a word that works once may not work in another writing.
- **Expect limits and prices in tokens, not letters.** [Why is my AI app bill higher than I expected?](https://spicyshark.co/learn/why-ai-bill-is-high/) shows what that does to a bill.
## Go deeper
*Optional. This part is for readers who write Python. The code is in the `code/` folder next to this lesson. `python3 show_numbers.py` prints the toy numbers below, and `python3 compare_tiktoken.py` prints the real-tokenizer numbers when the tiktoken package is installed.*
### What a model receives
Everything the model is given about a word is a list of integers. For `strawberry` the toy gives `[292, 114, 97, 119, 98, 263, 587]` and `o200k_base` gives `[302, 1618, 19772]`. In the toy, the ids 0 to 255 are single bytes, so 114 is the letter r, and each id from 256 up is a merge: id 256 + k is the pair that merge k joins. [What is a token?](https://spicyshark.co/learn/what-is-a-token/) builds the merges. A real table differs in size, not in kind: tiktoken 0.14.0 reports 100,277 ids for `cl100k_base` and 200,019 for `o200k_base`.
The same ten letters give different lists under different writings. This is the output of `compare_tiktoken.py`:
- `strawberry`: 496, 675, 15717 in `cl100k_base`; 302, 1618, 19772 in `o200k_base`.
- `Strawberry`: 2645, 675, 15717 in `cl100k_base`; 3504, 1134, 19772 in `o200k_base`.
- `STRAWBERRY`: 790, 22275, 33, 73995 in `cl100k_base`; 1117, 46176, 33, 132354 in `o200k_base`.
- a space, then `strawberry`: 73700 in `cl100k_base`; 101830 in `o200k_base`.
In `cl100k_base` the lowercase and capitalised words share the ids of `aw` and `berry` and differ only in the first chunk, 496 against 2645. A model has to learn that these lists name nearly the same letters.
### Why the count is not in the ids
The tokenizer's table maps an id to bytes, and a token model is not given the bytes. Two functions in `letters.py` count the same chunks in two ways. One looks inside each chunk. The other treats ids as letters and counts those equal to the letter's own id, since ids 0 to 255 are the single bytes:
```python
def count_from_chunks(text, letter, merges):
return sum(count_inside(spelling, letter) for _, spelling in chunks(text, merges))
def count_ids_that_are_the_letter(text, letter, merges):
wanted = {ord(letter.lower()), ord(letter.upper())}
return sum(1 for i, _ in chunks(text, merges) if i in wanted)
```
For `strawberry` and `r` the first returns 3 and the second 1. The second is what you get by reading ids as letters, and it misses every letter inside a bigger chunk. What an id means beyond that is learned, so a model must pick up the spelling of each chunk from training text. This is the kind of gap CUTE measures.
How often does a letter sit in a bigger chunk? On the toy's own training text, 2,231 words with 9,227 letters become 5,264 chunks, 1.75 letters per chunk, and 72.7% of all letters sit in a chunk of two or more letters. That is a property of this toy, 512 merges learned from 15,793 bytes of text. A production tokenizer has its own figure.
### A worked example by hand
Take a tiny table of three merges, applied in the order learned: id 256 is `e` then `r`, id 257 is `r` then `y`, and id 258 is `s` then `t`. Start from the ten bytes of `strawberry` and apply each merge to every place it fits:
| after | ids | chunks |
|---|---|---|
| bytes | 115 116 114 97 119 98 101 114 114 121 | 10 |
| merge 0 | 115 116 114 97 119 98 256 114 121 | 9 |
| merge 1 | 115 116 114 97 119 98 256 257 | 8 |
| merge 2 | 258 114 97 119 98 256 257 | 7 |
The model is given `258, 114, 97, 119, 98, 256, 257`, the same seven chunks the toy makes. Counting ids equal to 114 finds 1 r. Looking inside, the chunks `st`, `r`, `a`, `w`, `b`, `er` and `ry` hold 0, 1, 0, 0, 0, 1 and 1 r's: 3 in all. The test `test_worked_example_by_hand` checks each row of the table and the counts in this paragraph.
### Spelled out, in the toy and in tiktoken
`spelled_out("strawberry")` puts a hyphen between the letters. The toy's splitting pattern makes a hyphen a piece of its own, so the toy cuts `s-t-r-a-w-b-e-r-r-y` into 19 chunks: ten letters and nine hyphens. Both tiktoken encodings give 10 chunks, because the hyphen joins the letter after it, as in `-r`. A space between the letters does the same: ` r`. In both, the most letters in a chunk is 1, for the five words of the widget's list and both separators, and `compare_tiktoken.py` prints the check. In `o200k_base` the three r chunks are all id 6335. After spelling, counting a letter is counting the ids that name that letter.
### Models that read characters or bytes
If the chunks cause the trouble, a model could skip them. Several public papers try this, and each has to deal with a longer input: `strawberry` is 10 bytes and 3 `o200k_base` chunks.
- ByT5 (Xue et al.) takes text as its byte values, not pieces of a fixed vocabulary.
- CANINE (Clark et al.) reads text one character at a time, with no separate tokenization step.
- Charformer (Tay et al.) starts from characters and learns its own grouping into subword blocks during training.
- MEGABYTE (Yu et al.) groups a byte sequence into patches, with one component for the detail inside a patch and another for how patches relate.
- The Byte Latent Transformer (Pagnoni et al.) also groups bytes into patches, and sets the patch length by how predictable the next byte is. Its authors report that it matches models built on tokens at the scale they tested.
Zhang, Cao and You (2024), a preprint, note that models built for counting often use character-level tokens, while general language models use byte-level byte pair encoding, and report that results vary a great deal with how the input is tokenized. They also note that earlier work found writing out steps can help with counting. This lesson does not say how often any named model counts letters correctly today; it shows the input the model gets.
### Proof it works
```bash
cd lessons/why-ai-cant-count-letters/code && python3 -m pytest -q
python3 embed_widget.py --check
```
The tests check these claims. The hand example gives the ids above and counts 1 by ids against 3 by chunks. The lesson's own numbers are pinned: the seven chunks and ids of `strawberry`, the 2,231 words and the 72.7%. For the widget's five example words, counting inside the chunks equals `str.count`. For 300 random ASCII strings, the chunk spellings join back into the text and the count from chunks equals `str.count`. Spelled out with a hyphen or a space, no chunk holds two letters, in 200 random words each. Text that is not ASCII is refused rather than counted wrong, and more merges never give more chunks. The widget tests run the widget's JavaScript under Node.js (so they need it installed) on the ids and spellings of 506 strings at 0, 100 and 512 merges and require the same as the Python, and pin the sentences the widget writes. One more test pins every cut and id of the two published tokenizers printed above, and skips itself when tiktoken is not installed. The embed check fails if the merges or the starting word in `widget.html` drift from the code.
### Where this is used in practice
- **tiktoken** is an open-source tokenizer library under the MIT licence. `tiktoken.get_encoding("o200k_base")` returns an object whose `encode` and `decode` give the ids and the text back, which `compare_tiktoken.py` uses.
- **Code tools.** Providers run model-written Python in a sandbox, which is how a model can count letters by running `str.count` instead of recalling spellings.
## Skills this lesson builds
- Explain why a model is given token ids and not letters, and what that does to letter tasks.
- Cut a word into chunks by hand from a small merge table.
- Compare two tokenizers on the same words with a library and report where they differ.
- Choose between asking the model, asking it to spell first, and giving it a code tool, then test the choice on your words.
- Tell a measured claim about a model from a guess, by its date and its source.
**Portfolio project:** write a command-line tool that reads a list of words, prints their chunks and ids under two tokenizers, and flags each word where a given letter sits inside a chunk of two or more letters. Publish the table for 50 words of your own.
## Check yourself
1. In the toy, strawberry reaches the model as the ids 292, 114, 97, 119, 98, 263 and 587. How many r's can you find by looking for 114, and why is that not the answer?
Answer
One. The id 114 is a lone r, but two more r's are inside er (263) and ry (587), and nothing in those ids shows it. The true count is 3.Answer
Both tiktoken encodings then cut it into one chunk per letter, most with a hyphen, so no letter hides in a bigger chunk. In o200k_base every r chunk has the same id, 6335. How well a given model uses this is for you to test.Answer
The bytes 98 101 114 114 121 become 98 256 114 121 after merge 0 (e r) and 98 256 257 after merge 1 (r y); merge 2 does not apply. Looking for 114 finds none, and `berry` has two r's, one inside er and one inside ry.Answer
They share `aw` (675) and `berry` (15717) and differ only in the first chunk, `str` against `Str`. An id names a chunk including its capitals, so one capital letter gives a different number for that chunk, and the model must learn that both name nearly the same letters.Answer
The toy's splitting pattern makes each hyphen a piece of its own: 10 letters and 9 hyphens make 19 chunks. In tiktoken, the hyphen joins the letter after it, as in `-r`, so each letter and its hyphen are one chunk and there are 10.