About GPT-3 Token Counter
This page opens on p50k_base, the encoding of text-davinci-002 and 003 and the Codex models, with a code sample as the example. It is useful when you maintain a prompt written for those older completion models. It is free, needs no sign-up and runs in your browser, so nothing you enter is uploaded.
p50k_base adds tokens for runs of spaces so that indented code is cheaper than with r50k_base. Compared with the newer encodings it spends more tokens on non-English text.
How to use GPT-3 Token Counter
- Paste your prompt or text into the box, or tap a sample.
- Choose the encoding. o200k_base is for GPT-4o and newer, cl100k_base for GPT-4 and GPT-3.5 Turbo.
- Read the exact token count, characters, words and characters per token.
- Look at the coloured tokens below. Turn on Show token ids to see the numbers.
- To see an estimate for Claude, Gemini or Llama, pick the model in the estimate panel. It is labelled as an estimate.
- To work out a cost, type your price per 1 million tokens from your provider's pricing page.
Features
- Exact token counts with o200k_base, cl100k_base, p50k_base and r50k_base.
- Coloured token view, with the id of each token on hover and a switch to list all ids.
- Characters, words and characters-per-token ratio.
- Estimates for Claude, Gemini and Llama, labelled as estimates with the ratio shown.
- Cost calculator that uses the price you enter, so it never goes out of date.
- Handles other scripts and emoji, and groups tokens that split one character.
- The tokenizer loads on demand, so the page itself stays light.
Worked examples
function add(a, b) { return a + b } gives 13 tokens
One more than o200k_base and cl100k_base.
আমি ভাত খাই। gives 22 tokens
More than three times the 6 tokens of o200k_base.
Privacy: how it works
- Counting happens in your browser. Your text is not sent to Ulkaa or to any provider, and no model file is downloaded.
Tips
- Use the encoding that matches your model. The wrong one can be off by a large margin on non-English text.
- Trailing spaces and line breaks are tokens too. Trim a prompt before counting if you will trim it before sending.
- Keep a margin below a context limit, because the answer also uses tokens.
- A rule of thumb for English is about four characters or three quarters of a word per token, but check with the real count.
Frequently asked questions
Is it really free?
Yes. There is no price, no trial and no credit card. Ulkaa Tool does the work with your own device, so there is no server bill to pass on to you.
Do I need an account?
No. There is no sign-up and no login. Open the page and use it. An Ulkaa account is not needed for anything on this tool.
Does it run in my browser?
Yes. Your text are processed by code running in your browser tab, on your own device.
Is my data uploaded?
No. Your text are never sent to Ulkaa or to anyone else. You can check this yourself by watching the Network tab in your browser developer tools while you use the tool.
Which encoding did text-davinci-003 use?
p50k_base. This page opens on it.
More Token Counter pages
- Token CounterPaste a prompt to get an exact token count for OpenAI models, with every token shown.
- GPT-4o Token CounterAn exact token count for GPT-4o prompts, using the o200k_base encoding.
- ChatGPT Token CounterPaste a conversation and see how many tokens it uses, with cl100k_base.
- GPT-2 and Davinci Token CounterToken counts with r50k_base, the encoding of the original GPT-3 models.
- Claude Token CounterA labelled estimate of Claude tokens, next to an exact OpenAI count for comparison.
- Gemini Token CounterA labelled estimate of Gemini tokens for mixed-script text.
- Llama Token CounterA labelled estimate of Llama tokens for code and prose.