Due to Byte-Pair Encoding (BPE) character splits, Korean text generates 1.95× more tokens than equivalent English text. 1,000 Korean words consume ~2,599 tokens in DeepInfra — Llama 3.3 70B.
Relative to English baseline (1.0×).
Price to send 1,000 words of Korean text into context.
Discounted price for repeated Korean system context.
| Word Count Scale | Korean Tokens | Korean Input Cost | English Equivalent Cost | Tokenization Penalty |
|---|---|---|---|---|
| 1,000 words (Short Article) | 2,599 | $0.00026 | $0.000133 | +$0.000127 |
| 10,000 words (Whitepaper / Report) | 25,994 | $0.002599 | $0.001333 | +$0.001266 |
| 50,000 words (Book / Corpus) | 129,968 | $0.013 | $0.006665 | +$0.006332 |
| 100,000 words (Enterprise Repository) | 259,935 | $0.026 | $0.0133 | +$0.0127 |
Can consume almost 2× the tokens of English for equivalent semantic content.
Prompt caching is particularly impactful for Korean system prompts to offset the higher per-word token weight.
Most LLM tokenizers are primarily trained on English-heavy web datasets. Non-Latin characters in Korean (Hangul) split across multiple Byte-Pair Encoding (BPE) sub-word or multi-byte UTF-8 tokens, requiring approximately 1.95× more tokens to encode the exact same semantic meaning as English.
Prompt caching is particularly impactful for Korean system prompts to offset the higher per-word token weight. In addition, enabling prompt caching on static Korean instructions or documentation saves 75–90% on input token rates.
Yes. DeepInfra — Llama 3.3 70B has strong multilingual comprehension and generation capabilities in Korean (한국어). The difference is purely computational and financial due to sub-word token splits.