Simon Willison: Claude Sonnet 5 Has a Hidden Cost — New Tokenizer Produces ~30% More Tokens

Main idea

Claude Sonnet 5 is marketed as 60% cheaper than Fable 5 at $2/$10 per million tokens. Willison discovered that the new tokenizer produces ~30% more tokens for the same English text, significantly raising real costs compared to the marketing presentation.

Context

Willison published the analysis the day after Claude Sonnet 5 launched (June 30, 2026). It builds on his longstanding practice of testing LLM models and documenting their actual characteristics. Anthropic also dropped support for temperature, top_p, top_k — unusual for a mid-tier model.

Why it matters

For developers planning to migrate from Claude 3.5 Sonnet to Sonnet 5 based on pricing arguments, this is critical information. The actual cost per equivalent output may be closer to parity with Fable 5, not a 60% discount. Willison's analysis is a practical benchmark before making migration decisions.

Details / arguments

• New tokenizer: ~30% more tokens for English text
• Adaptive thinking enabled by default (generates extra reasoning tokens)
• temperature, top_p, top_k no longer supported — limitation for experimenters
• Promo prices $2/$10 valid until Aug 31, 2026 — post-promo economics worsen further
• Recommendation: test your own use-case costs before migrating