Tokenizer – Token Counter for GPT, Claude & Co.

Count tokens for any language model right in your browser. No API, no login: your text never leaves your computer. Exact tokenization for OpenAI models (GPT-5.x, GPT-4.1, GPT-4o, o-series, gpt-oss, GPT-4, GPT-3.5), an approximation for GPT-6 and estimates for Claude, Gemini, Llama, Mistral, DeepSeek, Qwen and Grok.

Tokens
0
Zeichen
0

What is a tokenizer?

A tokenizer splits text into tokens — the smallest units a language model understands. A token can be a whole word, a syllable, a punctuation mark or a single byte. GPT, Claude, Gemini & co. don't work with characters or words but with tokens. As a rule of thumb: 100 tokens ≈ 75 English words ≈ 400 characters. In German the ratio is somewhat lower because of long compound words.
‍

Why do I need a token counter?

A token counter is essential when working with LLMs:

  • Calculate API costs — OpenAI, Anthropic and Google bill per input and output token.
  • Stay within the context window — every model has a fixed limit that prompt, context and response together must not exceed.
  • Optimize prompts — cutting a prompt from 5,000 to 3,000 tokens saves 40% in costs.
  • Prepare fine-tuning data — every token counts in training.


How does tokenization work?

Modern LLMs use subword tokenization based on Byte Pair Encoding (BPE). Frequent character sequences are merged into one token, rare ones are split into smaller pieces.

Examples (o200k_base, e.g. GPT-5.x and GPT-4o):

  • Hallo Welt → 2 tokens
  • indivisible → 3 tokens
  • Donauschifffahrt → 6 tokens
  • 🙏🏾 → 3 tokens (emoji + skin tone modifier)


Supported models

Exact tokenization (OpenAI)

  • GPT-5.x, GPT-4.1, o1, o3, o4-mini (o200k_base)
  • GPT-4o, GPT-4o-mini (o200k_base) and gpt-oss (o200k_harmony)
  • GPT-4, GPT-4 Turbo, GPT-3.5-turbo (cl100k_base)
  • GPT-3 legacy (p50k_base, r50k_base)


Approximation and estimate

  • GPT-6 (approximated with o200k_base until OpenAI publishes the tokenizer)
  • Claude: Opus 4.7 and newer, Claude 5.x, Fable (new tokenizer) as well as Claude 3.5 to 4.6
  • Gemini 2.5 / 3.x, Llama 3.x / 4
  • Mistral, Mixtral
  • DeepSeek V3, R1 and newer
  • Qwen 2.5, 3 and newer
  • Grok 3, 4 and newer


Why an online tokenizer in the browser?

  • Privacy — no upload, no server requests, everything runs locally
  • Free — no login, no API key, no rate limit
  • Live — tokens are counted as you type


FAQ

What is the difference between characters and tokens?
Characters are individual letters. Tokens are word pieces that the model processes internally — on average about 4 characters per token in English and about 3.5 in German.

How do I count tokens for GPT-4?
Select the “GPT-4” model in the dropdown and paste your text — the token counter shows the exact number instantly.

Can I count tokens for Claude?
Anthropic does not publish the Claude tokenizer, but offers a free token counting API for exact numbers. Here we show an estimate with ±10–20% accuracy. Note: since Claude Opus 4.7, Claude produces around 30% more tokens for the same text than older models.

How many tokens are 1,000 words?
English: ~1,300–1,400 tokens. German: ~1,400–1,700 tokens.

How much do 1,000 tokens cost?
That depends on the model. With Claude Sonnet, for example, it is around $0.002 input and $0.01 output (as of October 2026). Output tokens are significantly more expensive than input tokens with almost every provider.

‍