Tokenization

Why can a model that writes working code not count the letters in a word?

It reads pieces, not letters.

You see banana as six letters: b, a, n, a, n, a. The model does not. Depending on the tokenizer, it sees something like ["ban", "ana"] or ["banana"]. The working unit is not a letter and not exactly a word. It is a token A piece of text from the tokenizer's vocabulary. It can be a whole word, a word fragment, a single character, or even part of a number. .

That difference sounds small until you ask a character-level question. “How many r’s are in strawberry?” is a count() call for you. For the model there are no separate r objects to iterate over. The letters are baked into token representations, and the answer has to be reconstructed rather than looked up.

Why not just use characters as tokens? Cost. internationalization is 20 characters, so a character-level model would spend 20+ tokens on one word. A subword tokenizer represents it in two or three pieces, which means shorter sequences, less compute, and cheaper training and inference. Tokenization is an efficiency decision first, and its side effect is that character-level work stops being natural.

A next-token machine has no character loop.

The architecture is a pipeline of one repeated question: given the tokens so far, what comes next?

text → tokens → embeddings → attention → hidden states → next-token probabilities

There is no built-in word.count("r"), no indexing into a string, no loop over characters. Those operations exist in code, not in the computation graph. The model can learn that the piece "berry" usually contains two r’s, but that knowledge is a statistical association stored in weights, not a procedure it can run on demand.

So the task “count the letters” is not a reasoning problem. It is a deterministic computation problem, and the model is the wrong machine for it.

Where this shows up in production.

The practical rule that follows: known workflow goes to code, uncertain reasoning goes to the model. Counting letters is a known workflow. So is reversing a string, finding the 7th character, or doing arithmetic. Summarizing, classifying intent, or deciding which tool to call is uncertain reasoning, and that is where the model earns its place.

This has a few everyday consequences:

• You pay per token, not per character. Billing and rate limits are token-based, and the same content costs different amounts in different languages.
• Context limits are token limits. A prompt that looks short in characters can be long in tokens, especially with rare words, code, or non-English text.
• Formatting changes the split. Extra spaces, punctuation and casing can move token boundaries, which is part of why small prompt changes sometimes produce surprisingly different behavior.
• The fix is a tool, not a better prompt. Expose a count_letters(text, letter) function through tool calling and the answer becomes exact, because the deterministic step moved to code.

The fine print.

• It is not that the model cannot see letters. It can, indirectly. It is that the operation is not native, so the answer is reconstructed from representations instead of counted.
• Correct answers do not mean the mechanism changed. A model that gets “strawberry” right may still fail “mississippi”, because each answer is a reconstruction, not a procedure.
• Longer and rarer means worse. Repeated letters, unusual names and long strings are the cases where reconstruction fails most often.
• Never estimate token counts by eye. English averages roughly four characters per token, but that average breaks on code, numbers, URLs, emojis and other languages. Measure with the actual tokenizer.
• Tokenizer changes are model changes. The same text tokenizes differently across models, so token counts, context budgets and costs do not transfer between providers.
• Do not ask the model to be a string library. Counting, reversing, indexing, slicing and arithmetic all have exact implementations one tool call away. Using the model for them is choosing the slower, less reliable path.