What is cached, and what is not
Every request to the model is one long sequence of text. It starts with the system prompt (the rules), then the tool list, then the skill list. In a multi-user set-up this opening block is the same for every request and is typically 10,000 tokens, roughly 7,000 words. We call it the prefix.
After the prefix comes the part that changes: the date and time, the user's name, the session id, the user's message, and then the conversation so far: answers, tool calls, tool results.
The model must read the whole sequence before it answers. Reading fresh text is the expensive part. Prompt caching lets the provider save its reading of the prefix and reuse it. A cached read costs about 10 percent of a fresh read on most models; on Fable 5.1 it is USD 0.25 per million tokens against USD 10.00, 40 times cheaper.
That gap is why the prefix deserves attention. It is the largest part of almost every request, and it is the part you control. With the right layout and lifetime, an organisation's total token spend falls by more than 70 percent; in the example under Rule 5 the prefix cost falls by over 90 percent.
Rule 1: the cache is shared across the organisation, not per session or per user
This is the one most people get wrong, and the one everything else depends on. The saved copy is not attached to a chat, and not to a user. The provider matches on the text itself and the model, within your organisation's API account.
If Maria sends a request whose first 10,000 tokens are the same as the request Alex sent five minutes ago, Maria's request reuses the copy that Alex's request created. She has never spoken to Alex. She is in a different session, on a different device. None of that matters.
So an organisation with twenty users does not have twenty caches. It has one, if the twenty requests start the same way. One person pays for the write; everyone reads.
Rule 2: keep the system prompt, the tool list and the skill list the same for all users, or the same within a group
Because the match is on the text, the prefix has to be identical byte for byte: same rules, same tools in the same order, same skills in the same order. A different word, an extra line, a reordered tool, and it is a different prefix with its own copy.
Each different prefix is written separately and kept alive separately. Twenty slightly different prefixes are twenty copies, each kept warm by one person. One shared prefix is one copy kept warm by twenty.
If two departments really need different rules, group them: one prefix for finance, one for sales. A few groups is fine. One prefix per person is the thing to avoid.
Rule 3: date, time, user name and session id go after the user message, never in the system prompt
A date or a user name in the system prompt is the most common way to break Rule 2 by accident. A name turns one copy into twenty. A date throws every copy away at midnight. Both look harmless in the prompt file.
Keep the values; move them. Put the date, the time zone, the user's name and the session id in the user's turn, after the shared prefix. We append a small block to each user message. The model still sees everything it needs. The prefix above is untouched.
Anything that differs between two users, or between today and tomorrow, goes below the prefix.
Editing the rules has the same effect as a variable. Save a changed instruction or a new skill description, and every request after it starts with new text, so the whole organisation writes a new copy. Once a day is fine. Ten times a day is waste. Batch the edits.
Rule 4: once the prefix breaks, nothing after it is cached
The cache does not work token by token or block by block. It works from the start of the request forward. The provider finds the longest saved copy that matches the beginning of your request and reuses that. From the first token that differs, everything is read fresh: the rest of the prompt, the tools, the skills, the whole conversation.
That is why the second panel of the figure above is red in every row. One date line in the first sentence does not cost you one line. It costs you the full 10,000 tokens, on every request, for everyone.
The same rule works in your favour when the prefix is intact. The copy covers the prefix and, within a session, the conversation so far, so a long session re-reads its own history at the cheap rate as well.
Rule 5: more than 10 users, change the default 5 minutes to 1 hour
A saved copy expires. By default it is kept for 5 minutes after the last use, and every use restarts the clock. After a gap longer than 5 minutes the next request pays for a fresh write.
Five minutes suits one person typing back and forth. An organisation does not work like that. People ask for a report, read it, take a call, go to lunch. In our October logs, 65 percent of all cache-write tokens were re-writes of text we had already paid for, after a pause longer than 5 minutes.
The 1-hour option costs more per write: 2 times the input price instead of 1.25 times. It pays for itself when the copy is read many times before it would otherwise expire. With ten or more active users, someone sends a request within the hour almost all the time, so the copy is written once in the morning and read for the rest of the day. Below ten users, measure your own gaps first.
How to check your own set-up
Every answer from the API comes with a usage count. Three numbers tell the story:
- cache_read_input_tokens should be the big number on almost every request. That is the prefix being reused.
- cache_creation_input_tokens should be large only on the first request of the day, and small after that. If it is large on every request, the prefix is changing between requests. Look for a date, a name, or a reordered list.
- With the 1-hour setting on, the creation count is reported as ephemeral_1h_input_tokens. If you only ever see the 5-minute kind, the setting is not taking effect. We found exactly that in our own transcripts: the option was set in the wrong place and silently ignored.
Prices in this post are the Fable 5.1 list prices on the day of writing.