Cogloom · Practical AI & coding

Prompt caching cut my Claude API bill by 85%

The exact setup — and the three gotchas that quietly undo it

Anthropic bills a cached input token at roughly a tenth of the normal input rate. If a big chunk of your prompt is identical on every call — a long system prompt, tool definitions, a fixed document — you are paying full price to send the same tokens thousands of times. Caching stops that.

Here is a real shape: an agent doing about 4,000 calls a day with a ~2,800-token system prompt (rules, tool definitions, examples) sent on every single call. That prefix is identical every time. Without caching you pay full input price for those 2,800 tokens 4,000 times a day. With caching, after the first write, every subsequent read of that prefix is billed at about a tenth of the rate — and on a prompt that is mostly the repeated prefix, the input bill drops by roughly 85–90%.

How it actually works

• You mark a prefix, not the whole prompt. Add "cache_control": {"type": "ephemeral"} to the last content block you want cached. Everything up to and including that block becomes the cached prefix.
• Reads are cheap, the first write is not. A cache write (the first time, or after expiry) costs more than a normal input token; a cache read costs a fraction of one. The saving comes from many reads amortising one write.
• It expires. The default cache lifetime is short (about five minutes), refreshed each time the prefix is hit; a longer TTL is available at a higher write multiplier. High-frequency traffic keeps it warm for free; a trickle of calls lets it lapse and you pay the write again.

The three gotchas

1. A minimum size. Prefixes below roughly a thousand tokens are not cached at all — caching a tiny system prompt does nothing. It pays off on long, stable prefixes.
2. Order matters — cache the stable part first. The cache matches from the start of the prompt. Put the unchanging content (system prompt, tools, fixed context) before anything that varies per request. One changed token near the front invalidates everything after it.
3. Watch the write multiplier on bursty traffic. If calls are spaced further apart than the TTL, every call becomes a write, and writes cost more than plain input. Caching then costs you money. It wins on frequency, not on novelty.

The one-line rule

Cache the biggest block of your prompt that is identical across calls, place it first, and only when you send it often enough to keep the cache warm. That is the whole game.