The Before/After: Same Task, Two Prompts
"Explain this error" with a clean 1,000-token prompt, vs the same request with the whole 100,000-token repository pasted in. Same model, same question — different bill:
Even the polite version — 25K tokens of relevant files instead of 100K — lands at ~$57,200/month on the same workload. The length tax is the least-visible line on any AI invoice because it looks like "context" rather than "spend."
The Length Tax Across Models
Per-request cost at three prompt sizes (1K / 25K / 100K input, 300 / 600 / 800 output), verified 2026-08-28 rates, no caching:
| Model | 1K prompt | 25K prompt | 100K prompt | 100K ÷ 1K |
|---|---|---|---|---|
| GPT-5.6 Sol | $0.0140 | $0.1430 | $0.5240 | 37× |
| GPT-5.6 Terra | $0.0056 | $0.0572 | $0.2096 | 37× |
| Claude Sonnet 5 | $0.0050 | $0.0560 | $0.2080 | 42× |
| Gemini 3.7 Flash | $0.0033 | $0.0411 | $0.1548 | 47× |
| DeepSeek V4 Flash (off-peak) | $0.00042 | $0.00590 | $0.02253 | 54× |
| GPT-5.6 Luna | $0.00056 | $0.00572 | $0.02096 | 37× |
The multiplier is stable across models (~37-54×) — the tax is proportional, so the mitigation is either shorter prompts or cheaper input. Caching flattens the curve dramatically when the long prefix repeats: a cached 100K prompt on DeepSeek bills at $0.0014 input instead of $0.022 — a 94% discount that makes long-context workflows viable.
When Long Prompts Earn Their Keep
The tax is real, but so is the value — on specific tasks. The deciding question is always: would a 5K-token summary produce the same answer?
- Whole-repo refactors with cross-file constraints
- Long-document legal or financial review
- Multi-file test generation with exact schemas
- Deep debugging where the stack trace spans files
- "Explain this error" with a pasted repo
- Summaries of documents a chunker handles
- QA over docs you could retrieve instead
- Any task where 80% of pasted tokens are never read
The Three-Lever Fix
Shrink: give the model the files that matter
Retrieve 5 files instead of pasting 30. Most "context" is never read — the model's attention is finite and the bill is linear.
Cache: keep the long prefix stable
If the 100K context repeats across requests (a shared codebase, a fixed corpus), cache it — 50-97% off the input line on most providers.
Route: send long-context jobs to cheap-input models
DeepSeek V4 Flash at $0.22-0.44/1M input makes 100K prompts affordable; sending them through a $5/1M flagship uncached is the single most expensive habit in this guide.
Frequently Asked Questions
How much does prompt length affect API cost?
Linearly on input — a 100,000-token prompt costs 100x a 1,000-token one before caching. At 1M requests a month on GPT-5.6 Terra that is $200 vs $20,000 just for input, plus 3-5x more on output if the longer context also lengthens answers.
Is a longer prompt always more expensive?
Yes on the input line, but caching changes the shape: repeated prefixes (system prompt + stable docs) bill at 10-97% off. The expensive pattern is per-user dynamic length — unique context that never hits the cache.
When does a 100K-token prompt actually pay off?
When the task genuinely needs the content: whole-repo refactors, long-document analysis, legal review. The test: would a 5K-token summary of the same material produce the same answer? If yes, the 100K prompt is 20x wasted spend.
How do I measure my prompt length cost?
Token calculator gives the per-prompt number; the prompt cost calculator prices the same prompt across models. Multiply by monthly request volume to see the real line item.
What is the cheapest way to use long context?
Route long-context jobs to models with cheap input or big cache discounts (DeepSeek V4 Flash, Gemini Flash tiers), cache the stable prefix, and keep dynamic content in the tail. Never send 100K tokens through a $5/1M input flagship uncached.