Guide

Count LLM Tokens: Estimates, Exact Counts and JSON Cost

Short answer: to count LLM tokens, remember that language models read tokens, not words, and you are limited and billed by them. Paste your text into the Token Counter to get exact characters and bytes straight away, a token estimate, and, if you choose, an exact count using the o200k_base or cl100k_base encodings.

Why tokens, not characters

A token is a chunk of text: sometimes a whole word, sometimes a piece of one. Context windows and prices are measured in tokens, so a prompt that looks short can still be expensive if it is full of symbols, code or unusual scripts.

Two rules of thumb

  • about 4 characters per token, for ordinary English;
  • about 0.75 words per token.

These are only rough guides. The counter shows exact characters and bytes, and the token figure is an estimate unless you switch on the exact encoder.

Exact counts

You can optionally load an exact tokenizer, o200k_base or cl100k_base. The tool says clearly which method produced the number. An exact count for one encoding is still not guaranteed to match every model, so treat it as the right ballpark for others.

Non-Latin text, including Urdu

Estimates are least reliable for non-Latin scripts, Urdu among them. The same sentence can need many more tokens than its length suggests. If you work in such a language, measure with the exact encoder, not the rule of thumb.

Why JSON is expensive

Every brace, quote and repeated key name costs tokens. The tool can show the same JSON in four layouts: compact, pretty, TOON and YAML (for up to 150,000 characters), so you can compare before you send. Pretty-printed JSON is the costliest to read for a model; compact and TOON layouts cut the repeated punctuation. Measure your own data, because the saving depends on its shape.

How to count LLM tokens with the Token Counter

  1. Open the Token Counter.
  2. Paste the text, up to 5 million characters.
  3. Read characters, bytes and the estimate.
  4. Turn on an exact encoding when the number matters.
  5. For JSON, compare the layouts and pick the smallest one your model handles well. If the JSON came from a model and will not parse, see fixing invalid JSON from ChatGPT and JSON unexpected token errors.

Limits to know

  • The default token figure is an estimate.
  • Different models tokenize differently.
  • Non-Latin text is the least reliable.
  • Everything runs in your browser.

Frequently asked questions

How many characters is a token? About four, for English.

How many tokens is 1,000 words? At 0.75 words per token, roughly 1,333.

What are o200k_base and cl100k_base? Two tokenizer encodings used by OpenAI models.

Is JSON or TOON cheaper? TOON usually has fewer tokens, but measure your own data.

Why is my Urdu text so costly? Non-Latin scripts often need more tokens per character.

Related tools and guides