The tokenizer tax

Every model price you’ve ever seen quoted - ‘$5 per million tokens’ - is measuring in a unit that differs between vendors. That’s not a nitpick. It quietly invalidates almost every cross-lab price comparison on the internet, including several we used to make.

Tokens, from zero

Models don’t read words; they read tokens - chunks of text, usually a few characters long, produced by a piece of software called a tokenizer. ‘MetaWatch is opinionated’ might be five tokens or eight, depending on whose tokenizer chopped it. You are billed per token, in and out.

Here’s the problem: every lab ships its own tokenizer, and they are not the same size. Anthropic’s own documentation concedes its current tokenizer can use up to 35% more tokens for identical text than its predecessor - and one measurement on a real TypeScript file put Claude’s count at 1.73× GPT-5.x’s for the same content. Same file, same job, 73% more billable units. A ‘$5 per million’ sticker tells you almost nothing until you know how big the million is.

Where it really bites: reasoning tokens

Modern models ‘think’ before answering - and the thinking is billed as output. Two models with identical stickers can differ several-fold in how many reasoning tokens they burn per task. This is why Theo’s read on the Opus 5 launch matters: the headline said half Fable’s price, but measured against real reasoning-token volumes it’s ‘closer to like 20 to 25% off’, because ‘the smartest models use less’. The community arrived at the same place independently - the top-voted framing on Reddit was shrinkflation: ‘newer models just burn through vastly more tokens, often enough with similar results… They can’t increase the price and they know it, so they make their models think more.’

The two numbers that are honest

• Cost per task. What did it actually cost to complete a unit of real work, all tokens included? This is the only cross-lab comparable number, and the reason our board leans on figures like Grok 4.5’s $0.31 per benchmark task rather than its sticker.
• Cached-input price. Agents resend their accumulated context at every step of a loop, and vendors charge a deep discount for input they’ve seen before - Gemini 3.6 Flash caches at $0.15 against a $1.50 sticker, a 10× discount. For agentic workloads, the cache price often dominates the whole bill.

And one number that’s honest but hidden: subsidy. Subscriptions (Claude Max, Codex, Cursor) sell tokens below cost at multiples that dwarf any sticker difference. If your usage rides a subscription, your effective price is the subsidy, not the menu. If you’re serving a product over the API, the menu is real - which is why our Serving picks and our subscription advice sometimes disagree.

How to read the next pricing announcement

1. Ignore the per-token sticker until you know the tokenizer and the reasoning-token appetite.
2. Ask for cost-per-task. If the vendor won’t say, someone at Artificial Analysis has measured it.
3. Check the cache price if you’re building agents. It’s probably your real bill.
4. Assume today’s price is a promotion. In one 60-day window this summer, three labs changed terms under their customers’ feet.

Related: Parameters, and big-model smell for why the expensive models exist at all, and The benchmarks worth believing for how we treat the numbers vendors do publish.