How LLM prompt caching works, how each provider implements it, when it saves money, and how to calculate your actual savings before writing a line of code.
Every LLM API call begins with an attention computation — the model processes all input tokens to build a key-value (KV) cache before generating any output. This is the expensive part. When the same prefix appears in a subsequent request, the provider can skip this computation and reuse the stored KV cache from disk.
The key insight: caching only works on a prefix. Everything before the first dynamic token is cacheable. Everything after the first change is not. This is why moving dynamic content (user messages, timestamps) to the end of your prompt is critical.
| Provider / Model | Discount | Min Cache Size | Mode | Cached $/1M | Full $/1M |
|---|---|---|---|---|---|
| OpenAI (GPT-5.6 Terra) | 75% off input | 1,024 tokens | Automatic | $0.50 | $2.00 |
| Anthropic (Claude Sonnet 5) | 90% off input | 1,024 tokens | Manual breakpoints | $0.20 | $2.00 |
| Google (Gemini 3.6 Flash) | 75% off input | 2,048 tokens | Automatic | $0.375 | $1.50 |
| DeepSeek (V4 Pro) | ~90% off input (off-peak) | N/A | Automatic | $0.07 | $0.50 |
| xAI (Grok 4.6) | N/A | N/A | Not published | — | $3.00 |
| Mistral (Large 3) | N/A | N/A | Not published | — | $2.00 |
Prices in USD per 1M input tokens. Verify current rates at each provider's pricing page.
Caching saves money proportional to how much of your total input is in the cacheable prefix. Use this formula:
Worked example: Customer support bot
Prompt caching is a provider-side optimization where the model's KV (key-value) attention cache is stored between requests. When a new request starts with an identical prefix to a cached one, the provider reuses the cached computation instead of reprocessing those tokens — charging you the cheaper cached-input rate instead of the full input rate.
No. Prompt caching is computationally equivalent to a full forward pass — the model produces identical outputs. Only the billing changes. The cache is deterministic on the exact token sequence, so as long as your prefix is unchanged, you get the same quality at a lower price.
On OpenAI, prompt caching is automatic for supported models. You don't need to change your API calls — the system automatically detects repeated prefixes. You can see cache hit statistics in the usage object returned with each API response (usage.prompt_tokens_details.cached_tokens).
Anthropic uses explicit cache breakpoints — you mark specific positions in your message array with a cache_control annotation. This gives you precise control over what gets cached but requires code changes. The discount is 90% off input tokens (vs OpenAI's 75%), and the minimum cacheable block is 1,024 tokens.
Common reasons: (1) Your prefix changes on every request (e.g., you include a timestamp or unique ID early in the system prompt). (2) Your cached content is shorter than the provider minimum (1,024–2,048 tokens). (3) You're using a model or endpoint that doesn't support caching. (4) Cache entries expired — most providers cache for 5–10 minutes of inactivity.