Prompt caching

When many requests share a long prefix, such as a brand guide, a design system, or a big document, the provider can cache that prefix and bill repeat reads at a fraction of the input price. Gopura passes your caching instructions through, keeps related requests on the same provider, and reports every cache read and write in usage.

Which models cache

• Automatically, no markers needed: OpenAI, Google Gemini, DeepSeek, and most other providers. Repeated prefixes are cached when they are long enough.
• Only when asked: Anthropic (Claude), MiniMax and Alibaba (Qwen). These models cache nothing unless you add a marker or turn on automatic caching (below).

Cache prices for each model are in the catalog (GET /v1/models): pricing.input_cache_read and pricing.input_cache_write, per token, with margin included. For Claude, a cache read costs 10% of the input price and a cache write costs 125%, or 200% with a one-hour lifetime. input_cache_write is the five-minute price.

Explicit markers

Add cache_control to the message, or content part, that ends the stable prefix. Everything up to and including it is cached:

{
  "model": "anthropic/claude-sonnet-5",
  "messages": [
    {
      "role": "system",
      "content": [
        { "type": "text", "text": "<your 20-page brand guide>", "cache_control": { "type": "ephemeral" } }
      ]
    },
    { "role": "user", "content": "Write three taglines for the autumn campaign." }
  ]
}

A message-level marker works too: {"role": "system", "content": "…", "cache_control": {"type": "ephemeral"}}. Anthropic allows up to four markers per request, and caches entries for five minutes, refreshed on every hit.

One-hour cache lifetime

If requests that share a prefix arrive more than five minutes apart, such as a batch job that pauses between runs or a person who reads before replying, the five-minute cache expires between them. Add "ttl": "1h" to the marker to keep the entry for an hour:

{ "type": "text", "text": "<your 20-page brand guide>", "cache_control": { "type": "ephemeral", "ttl": "1h" } }

A one-hour write costs 200% of the input price instead of 125%. Reads cost 10% either way. One-hour caching pays off when the prefix is read again after a five-minute entry would have expired.

You can mix lifetimes in one request, but one-hour markers must come before five-minute ones. ttl applies to explicit markers only. Automatic caching (below) always uses the five-minute lifetime.

Automatic caching

Put cache_control at the top level of the request and the gateway places the markers for you: on the latest message, and on the message before the last user turn. Multi-turn conversations and agent loops then cache well without any per-message bookkeeping:

{
  "model": "anthropic/claude-sonnet-5",
  "cache_control": { "type": "ephemeral" },
  "messages": [ … ]
}

Automatic caching is off by default. A cache write costs more than plain input, so it only pays off when a later request reads the cache. For strictly one-shot traffic, leave caching off.

Keep related requests together: session_id

A model can be served by more than one provider, and each provider keeps its own cache. Send the same session_id with related requests, such as one conversation, one agent run, or one batch over the same document, and they stay on the same provider, so the cache you paid to write gets read:

{ "model": "anthropic/claude-sonnet-5", "session_id": "brand-run-42", "messages": [ … ] }

You can also send it as the x-session-id header. If neither is set, OpenAI's prompt_cache_key is used. Use an opaque value of up to 256 characters, not personal data. The gateway hashes it, scoped to your account, before anything leaves Gopura.

Verifying hits

Every response reports cache activity in usage.prompt_tokens_details:

"usage": {
  "prompt_tokens": 7786,
  "completion_tokens": 11,
  "prompt_tokens_details": { "cached_tokens": 7770, "cache_write_tokens": 0 },
  "cost": 0.00106
}

• cached_tokens are tokens read from the cache, billed at the cache-read price.
• cache_write_tokens are tokens written to the cache on this request, billed at the cache-write price, or the one-hour price for writes under a ttl: "1h" marker.
• Both are included in prompt_tokens. usage.cost is the exact total.

The first request with a new prefix shows cache_write_tokens; a repeat within the cache lifetime shows cached_tokens. If repeats show neither, check three things: the prefix is long enough (Claude's minimum is 1,024 to 4,096 tokens, depending on the model); nothing before the marker changes between requests, such as a timestamp in the system prompt; and the requests share a session_id.