Study Finds Non-English Languages Cost More to Process in AI Models
A study reported by Rest of World found that some non-English languages require 1.5 to 5 times more AI tokens than the cheapest languages, creating a "tokenizer tax" that raises costs and reduces usable context for speakers of those languages.
2newsrooms
2articles
26 hoursrunning
Oct 26first seen
Ask Scout
Scout answers questions about this story using only the coverage below.
Coverage over time
Articles
| Oct 27, 5:30 AM | techmeme.com | Study: non-English languages can consume far more AI tokens, with the most expensive using 1.5x-5x more than the cheapest, resulting in a "tokenizer tax" (Ananya Bhattacharya/Rest of World) |
| Oct 26, 3:00 AM | restofworld.org | AI models cost more when you don’t use English |