Due to Byte-Pair Encoding (BPE) character splits, Spanish text generates 1.15× more tokens than equivalent English text. 1,000 Spanish words consume ~1,533 tokens in Cerebras — GPT OSS 120B.
Relative to English baseline (1.0×).
Price to send 1,000 words of Spanish text into context.
Discounted price for repeated Spanish system context.
| Word Count Scale | Spanish Tokens | Spanish Input Cost | English Equivalent Cost | Tokenization Penalty |
|---|---|---|---|---|
| 1,000 words (Short Article) | 1,533 | $0.000537 | $0.000467 | +$0.000070 |
| 10,000 words (Whitepaper / Report) | 15,329 | $0.005365 | $0.004665 | +$0.0007 |
| 50,000 words (Book / Corpus) | 76,648 | $0.0268 | $0.0233 | +$0.003499 |
| 100,000 words (Enterprise Repository) | 153,295 | $0.0537 | $0.0467 | +$0.006998 |
Verb conjugation suffixes and gender agreements produce ~15% more tokens than concise English phrasing.
Latin script languages perform consistently well across all major tokenizers with minimal cost penalty.
Most LLM tokenizers are primarily trained on English-heavy web datasets. Non-Latin characters in Spanish (Latin) split across multiple Byte-Pair Encoding (BPE) sub-word or multi-byte UTF-8 tokens, requiring approximately 1.15× more tokens to encode the exact same semantic meaning as English.
Latin script languages perform consistently well across all major tokenizers with minimal cost penalty. In addition, enabling prompt caching on static Spanish instructions or documentation saves 75–90% on input token rates.
Yes. Cerebras — GPT OSS 120B has strong multilingual comprehension and generation capabilities in Spanish (Español). The difference is purely computational and financial due to sub-word token splits.