OpenLuxWhat is prompt caching? How it works on Claude, GPT and Gemini
Prompt caching reuses a previously processed prompt prefix so repeated input tokens can be served at a lower cache-read rate. It helps when a long, stable context is called repeatedly; it does not help when every prompt is unique or when stale context lowers answer quality.
Reviewed by OpenLux Engineering Team · Last verified
How prompt caching differs across Claude, OpenAI and Gemini
Provider implementations and billable units change. Verify the current official documentation and invoice fields before using these concepts in a cost model.
| Provider | Typical cache model | What to verify |
|---|---|---|
| Anthropic Claude | Explicit cache breakpoints on supported prompt sections | Minimum cacheable length, write/read pricing, supported TTLs and model coverage |
| OpenAI | Automatic prefix caching for eligible requests | Minimum prompt length, cache granularity, retention behaviour and discounted cached-input billing |
| Google Gemini | Implicit or explicit context caching depending on model and API | Cache storage cost, TTL, minimum tokens, model support and whether the cached content can be shared |
| Gateway view | May normalise access but cannot assume identical provider cache semantics | Whether the gateway exposes cache usage, upstream model identity and cache-specific billing fields |
How we evaluated this
This guide explains the caching concepts, not a universal rate card. It uses the current public documentation from Anthropic, OpenAI and Google on 3 November 2026, and separates cache economics from semantic caching and ordinary application caching.
OpenLux currently publishes a catalogue rate for each model. This page does not claim that every upstream cache discount is separately exposed in the OpenLux catalogue or console. Use the authoritative provider documentation and final invoice fields when a cache decision materially changes spend.
When should you test prompt caching?
A cache experiment is worthwhile when the same substantial prefix is reused often enough to offset cache writes, storage and added complexity.
- Measure the stable prefix separately from user-specific instructions and retrieved context.
- Confirm the selected model supports the required cache mode and minimum length.
- Measure cache writes, cache reads, misses and evictions on the same workload.
- Compare cost per successful task, not cache-hit rate alone.
- Test whether stale or incorrectly ordered context changes answer quality.
- Include cache storage or write costs in the total monthly calculation.
- Keep a no-cache control group for representative prompts.
- Document retention, privacy and cross-request cache isolation requirements.
What is prompt caching?
Prompt caching stores reusable model input state or a processed prompt prefix so a later request does not need to process every token from scratch. The provider can charge a cache-write or storage cost and then bill matching cached input at a different cache-read rate.
The benefit comes from repetition. A long system prompt, repository instruction file or shared document can be reused across many calls. A prompt that changes in its opening tokens may miss the cache because the stable prefix is no longer the same.
What is the difference between prompt caching and semantic caching?
Prompt caching usually keys on an exact or provider-defined prefix, so it accelerates repeated processing of the same context. Semantic caching stores a previous answer and retrieves it when a new prompt is considered similar. The first improves the model call; the second may bypass the model call entirely.
Semantic caching adds a similarity threshold and answer-reuse risk. Two questions can look similar but require different answers because of dates, account permissions or subtle constraints. Use it only for low-risk, well-bounded tasks and test false hits before production.
How does prompt caching work on Claude, GPT and Gemini?
Claude supports explicit cache breakpoints on supported prompt sections. OpenAI applies automatic prefix caching to eligible prompts. Gemini offers context caching with model- and API-specific modes. The details are not interchangeable: minimum sizes, TTLs, pricing fields and controls differ.
Design the prompt so the stable content comes first and the changing request comes last. Then test the exact model and provider path. A gateway that routes between providers may change cache behaviour even when the public request shape stays the same.
When does prompt caching save money?
Caching becomes valuable when the reused prefix is long, repeated often and billed at a meaningful cache-read discount. It is less useful when prompts are short, unique or dominated by output tokens. A cache write that is never reused is an extra cost rather than a saving.
Measure break-even reuse. Divide the added write or storage cost by the per-call saving after the first cache read. If the workload does not cross that reuse threshold in the retention window, keep the simpler no-cache path.
How do you calculate effective LLM cost with caching?
Build the bill from fresh input, cache writes, cache reads, output and retries. Do not multiply the headline input rate by all input tokens when the provider exposes multiple cache-specific rates. The same request can contain uncached, written and read tokens at the same time.
Reconcile the formula with actual usage fields and the invoice. Provider pricing pages can change and a percentage discount is not enough if the cache write cost or retention window makes the workload unsuitable.
How does caching affect coding agents?
Coding agents repeatedly send instructions, repository maps and tool definitions before the newest task. That stable prefix is a natural caching candidate, but long sessions also accumulate changing history and tool output. Measure the stable and changing portions separately.
A higher cache-hit rate is not a goal by itself. If the agent repeats stale context or makes more tool calls, the saved input cost can be offset by rework. Compare accepted tasks, tool calls, output tokens and human review with and without caching.
What should you monitor after enabling caching?
Track cache write tokens, read tokens, misses, hit ratio, time to first token, total cost per successful task and answer quality. Segment by model, endpoint, application and stable-prefix version so a cache regression does not disappear inside one blended total.
Version the cached context and invalidate it when policy, pricing data or product documentation changes. A cache that improves latency but serves outdated instructions is not a successful optimization.
Effective cost formula
Use the provider's actual billable fields; the labels below are conceptual, not a universal rate card.
Cost per workload
effective_cost =
uncached_input_tokens * input_rate
+ cache_write_tokens * cache_write_rate
+ cache_read_tokens * cache_read_rate
+ output_tokens * output_rate
+ retry_cost
+ cache_storage_cost
# Decision metric
cost_per_successful_task =
effective_cost / accepted_tasksModels with long-context or agent workloads
| Model | Vendor | Price | Vs. official |
|---|---|---|---|
| claude-sonnet-4-5-20250929 | Anthropic | $0.2647 / 1M tokens | -91% |
| gpt-6-astra | OpenAI | $0.3677 / 1M tokens | -96% |
| gemini-3.8-flash | $0.0551 / 1M tokens | -92% | |
| deepseek-v4 | DeepSeek | $0.0613 / 1M tokens | -90% |
Prices are metered in USD; the full list lives in the model catalogue.
Key takeaways
- Prompt caching discounts repeated prefixes; it does not make unique prompts free.
- Semantic caching is a different mechanism that may bypass the model and carries answer-reuse risk.
- Claude, OpenAI and Gemini expose different cache controls, rates and retention rules.
- Evaluate cache savings on cost per accepted task, not hit ratio alone.
- Reconcile cache usage with actual provider fields and the final invoice.
Frequently asked questions
What is prompt caching in simple terms?
It lets a provider reuse a previously processed prompt prefix, reducing the work and billable rate for repeated context. It works best when a long, stable prefix is reused across multiple requests.
Is prompt caching the same as semantic caching?
No. Prompt caching reuses the processed prompt context. Semantic caching stores prior answers and retrieves them for similar questions, which can skip the model call but risks returning an answer to a question that only looks similar.
Does caching always reduce LLM cost?
No. Cache writes, storage, misses and low reuse can outweigh the discount. Prompts dominated by unique input or output tokens may see little benefit.
How does Claude prompt caching differ from OpenAI?
Claude supports explicit cache breakpoints on supported content, while OpenAI describes automatic prefix caching for eligible requests. Verify current minimums, TTLs, model support and billing fields rather than assuming the same behaviour.
Does OpenLux apply the same cache rate to every model?
OpenLux publishes a catalogue rate and does not document one universal cache discount across providers. Do not infer cache-specific pricing without checking the current model documentation, console usage fields and invoice.
Which workload benefits most from prompt caching?
A repeated long context with small changing user instructions is the clearest candidate. Examples include coding agents with stable instructions, document analysis over the same source and support systems with a fixed policy prefix.
Sources and official documentation
- Anthropic prompt cachingOfficial cache controls, eligibility and usage fields.
- OpenAI prompt cachingOfficial automatic prefix-cache behaviour.
- Gemini context cachingOfficial Gemini context-cache modes and pricing considerations.
- OpenLux AI cost optimizationHow caching fits the broader cost-per-task framework.
- OpenLux model catalogueCurrent published model IDs and catalogue rates.
Measure cache savings on a real workload
Run a controlled cache and no-cache test, reconcile usage fields with the invoice, then compare cost per accepted task.