See exactly how a model reads your text.
Type or paste text and watch it split into tokens live — with a token count, the raw token IDs, and the boundaries highlighted right on the text. Nothing is ever uploaded.
Token boundaries
Each shaded span is one token. Colors just alternate so adjacent tokens are easy to tell apart — they don't mean anything else.
Token IDs
- 1
Paste or type your text
Results update live as you type — nothing to submit.
- 2
Pick a model
Switch between GPT-4o and GPT-3.5 / GPT-4 tokenizers.
- 3
Read the breakdown
See the token count, the boundaries, and the raw IDs.
Good to know
Is my text uploaded anywhere?
No. Tokenizing runs entirely in JavaScript in your browser tab using the open-source gpt-tokenizer library. Your text never leaves your device, and this page makes no network requests to do its work.
Why do different models split text differently?
Each model family uses a byte-pair encoding vocabulary trained on its own data, with its own set of learned merges. GPT-4o'so200k_base encoding and GPT-3.5/GPT-4'scl100k_base encoding have different vocabularies and vocabulary sizes, so the exact same text can come out as a different number of tokens with different boundaries in each.
What's a token, roughly?
A token is a chunk of text a model treats as one unit — sometimes a whole common word, sometimes part of a longer or rarer word, a punctuation mark, or a few bytes of a character from a non-Latin script or an emoji. Tokens come from a fixed vocabulary built by repeatedly merging the most frequent adjacent byte pairs seen in a large training corpus, not from a rule like "one token per word."
Is this the same tokenizer the real APIs use?
Yes, for these two encodings. gpt-tokenizerreimplements the actual cl100k_base ando200k_base vocabularies and merge rules used by OpenAI's tiktoken library, so token counts here should match what those APIs report for the same text.