Tokens-per-Word Chart by Model

Type or paste text; see how many tokens each model's tokenizer produces, side by side. Useful when deciding which model to use for non-English content (where tokenization efficiency varies wildly) and for estimating cost impact of switching models for a fixed text corpus.

How to use the Tokens-per-Word Chart by Model

Type or paste a text sample. The tool estimates how many tokens each model's tokenizer would produce and charts them side by side, so you can compare efficiency across English, multilingual, and code inputs. Use the sample buttons to load ready-made English, multilingual, and code text.

Why tokenizers split text differently

Every model breaks text into tokens with its own tokenizer, trained on its own vocabulary. For English, most land near three-quarters of a word per token, but the ratios diverge sharply on other content: a tokenizer with little non-English training data splits those languages into many more tokens, and code with lots of punctuation and indentation fragments differently again.

Since you pay per token and context windows are measured in tokens, those differences are real money and real capacity. This chart estimates the token count each model's tokenizer produces for the same sample, side by side, so you can see which model handles your kind of text most efficiently. For exact counts, use the OpenAI token counter or the Claude token counter.

Common use cases

• Multilingual planning — see which model tokenizes a non-English language most efficiently.
• Cost comparison — estimate how switching models changes token count for a fixed corpus.
• Context budgeting — gauge how much of a text fits a given context window.
• Code vs prose — compare how tokenizers handle source code against plain text.
• Model selection — factor tokenization efficiency into a choice, not just the per-token price.