How to pay less money by caching your prompts

Contact: blondeslutappreciator@proton.me

Up to date as of October 2026. Full credits to: https://rentry.org/pay-the-piper-less.

TL;DR

$0.41 per message: a 100k-token chat on Opus 5.5 with a 400-token reply.

$0.04 per message: the same message, but with caching.

This works just as well for GPT, Gemini, DeepSeek, Kimi and GLM and lets you save 80–97% on input cost per message.

Nice side effect: Your answers will also come back faster if you cache, as the provider won't have to reprocess the start of your prompt every request.

Are there any downsides? Non-Claude models: Nope, zero downsides or risks. Claude: Yes, you could double costs in the absolute worst case, if you are (a) sending less than one message per hour; (b) somehow fuck it up really really super badly. But just read the guide and I don't see any way you could be losing money with this.

Step by step:

0. Optional: Get my Companion Extension. I may be biased as the guy that made it, but it'll hold your hand, and show you any issues, both during setup, and during later use.

1. If you're on Claude:
   Base setup like I run it: open config.yaml in your SillyTavern main folder, NOT the one in the \default folder, and set these values:
   

   ⎗

   ✓

   1 2 3 4

   claude: enableSystemPromptCache: true # free extra backup cache point after the system prompt cachingAtDepth: 4 # depth to put the cache point at extendedTTL: true # true=1 hour TTL instead of 5 minutes TTL

   Details on each: enableSystemPromptCache, cachingAtDepth, extendedTTL.

   Restart the SillyTavern server.

2. If you're on OpenRouter: pin one provider in the model settings, as they do NOT share cache between each other.
3. Read Rules for everyone, then your model's section: Claude or Other models.
4. Press "Check caching" in the extension and fix any issues it shows you.

What caching is

The cache is everything from the start of the prompt to (Claude: The marker SillyTavern sets; Other models: Automatically set, making it easy for you, but also easy to cache suboptimally). Is it the same? It caches and you save money. Are there differences? You'll either cache badly or not at all.

Imagine what you send is ABCD. The provider calculates, then stores that ABCD for you.

• Send ABCDE next: ABCD is read from the cache cheap, E is added at the write price.
• Send ABCDEF after that: same thing, ABCDE cheap, F added.
• Send ABCDEF again (swiping): ABCDEF is read from the cache cheap. Maximum savings.
• Send BCD, or ABXD: nothing matches as the provider only stored ABCD as a whole, not AB. You pay the write price for everything again.
• Send ABCD again: ABCD is read from cache cheap. Each piece of stored cache is fully independent, so you can go back and forth as you like, as long as that specific cache hasn't exceeded its TTL yet.

TTL (cache lifetime): how long the provider keeps the cached message. How long that is depends on the provider, but is almost always between 5 and 60 minutes. See the tables under Claude and Other models. This matters because keeping cache saved costs the provider money, so they will delete unused cache after a while. Every message you send that uses a cached message resets the expiry timer on it. Yes, that means that if you do 3 hours of roleplay, always answer/swipe in time to refresh expiry, and don't break cache (switching preset/chat/etc), then you may only have to write each token once(!). For a nice extension that refreshes TTL for cheap see Extras.

Rules for everyone

1. Summarize. Caching and summarization go hand in hand. If you do not summarize, your RP will go bad very quickly. See Summaries.
2. Context size higher than you'll ever use. The problem is that as soon as you hit max context size, SillyTavern will start dropping messages one by one. No message will cache anymore, and it will also not tell you about the issue. You should still never actually use the full context, as LLMs go dumb the more you feed them. See Summaries.
3. No changes above the chat history. Take special care with macros like {{random}}, {{roll}}, {{time}}, {{date}}, and others. They're quite common in cards and presets, and will invalidate your cache on almost every message. That does not mean they are forbidden though. You just need to make sure they are at, or near, the bottom of context.

Force invalidate cache:

• Toggling preset entries, switching presets, editing the card. Pretty much guaranteed to invalidate the whole cache. Nothing that should keep you from doing it, but keep in mind that each switch will cost a full write. If you have toggles you use often, consider putting them in a low-depth injection instead.
• Editing, hiding or deleting old messages far up in context; adding a summary.
• Switching provider/model/effort. Each provider or model has separate caching. Effort can also force invalidate cache depending on model.

Keep in mind that every cache is standalone. If you chat with model A, switch to model B for a few messages, then go back to model A, your model A cache is still there as long as its TTL has not passed.

Things that are bad:

• Group chats. Are also super bad even if you ignore caching. Just copy all card defs into a new card and you'll have way more fun.
• The Summarize extension included in SillyTavern. Completely fucks shit up, and it's also horrible and bad even if we ignore caching.

Lorebooks

Lorebooks still work fine, BUT most of them need adjustment.

They're from an age before caching was relevant, so most authors just did not set them up to be cache compatible. That means you should check them, easily done with the extension. If there are issues, you can fix most of them if you invest a bit of time.

Disclaimer: I am not really a lorebook user. Below is MY interpretation of this, which works, but may not be ideal.

You can break it down into two basic types of entries:

Always needed (world info, main characters, rules, anything that triggers often): constant (the blue circle), position ↑Char or ↓Char, or anywhere else static above the chat history. This will permanently add them into context, meaning they'll be fully cached and cheap, but bloat context.

Rarely needed (a minor character, small locations, anything that comes up once or twice only): normal (green circle), position @D, depth 0–3. This will only add them into chat when they're triggered, meaning no caching for their tokens, but also avoids bloat. Really only worth it if an entry triggers <5% of the time.

The @D depth on "normal" conditional entries needs special attention. On non-Claude it will prevent caching of all messages below it the whole time it is triggered, so the higher it is the more money you lose. On Claude it instead completely breaks caching if it isn't shallower than your cachingAtDepth, so 0–3 with the setup above. (This author also sees relatively little reason for an insert depth higher than 3 for conditional entries.)

Summaries

You must still summarize and cut old parts of the RP regularly. Caching makes it easy to be lazy here due to lower $costs, but if you stop summarizing your RP will go bad quickly. The reason is that every LLM gets dumber and dumber the more context it has to handle.

Examples:

• Characters ignoring their definitions and behaving like stereotypes instead
• Char A knowing what only char B should
• Looping of everything. Be it story points, 'isms, drama, tics, sentence constructs, always ending/starting a reply with the same thing, and a lot more.

Just imagine that every x thousand tokens in context equal a few lost IQ points. If you notice the RP going bad, try a summary.

How to summarize

What I do is write one big text with all the important things I want the AI to remember. I then put that text into context above the chat history. You can do that via preset, via a lorebook, or other methods. The important thing is to have it be above the chat, so it stays constant, and cache keeps working. I then /hide all old messages but the last few as a seed. If you can, try to pick a natural breakpoint in the story for the message you leave visible. For example, for an RP at message 170 with a new day starting at 156, I would summarize it fully, then do /hide 0-155.

Extensions

I do it all by hand, with a lot of AI assistance, but there are also a lot of extensions out there, and people seem pretty happy with them. Except for the default ST summary extension, I cannot warn against that one enough, it's terrible.

Here are some extensions that were recommended to me. There are a lot of others out there, please don't take this list as exhaustive:

• https://github.com/Kristyku/InlineSummary
• https://github.com/vadash/Extension-Summaryception
• https://gitgud.io/Monblant/extblocks

But how well do they cache?

• InlineSummary: works fine. You manually pick a range of messages, and it replaces them with a single summary message, same as doing it by hand.
• Summaryception (the original by Lodactio): completely kills cache. Use the fork below.
• Summaryception (vadash's fork): needs you to configure it, or it'll work badly. It's not hard though.
  • Memory Mode: Set to Prefix Cache, this makes it summarize every 18k token of chat. Default would summarize every 3 to 8 messages which is horrible.
  • Placement: Macro Only, then put {{summaryception_memory}} in a preset entry directly above Chat History. Yes, you can place it elsewhere, but this is the best spot for it, both from a preset and a caching perspective. Default placement has no advantage, but re-bills your whole preset every time it summarizes.
• Vector Storage (built into SillyTavern): "Enabled for chat messages" kills the cache. It reshuffles your prompt history which is the most anti-cache thing there is. The "Include hidden messages" does nothing since an update in March 2026 (pull request 5378) botched it. For files or lorebooks it only injects so those work, but keep the at a low depth.
• ExtBlocks: you need to be careful, then it works fine. Each block is regenerated on every (or every few) user messages. If you inject it at a low enough depth it works fine, if you inject it too high you kill cache.

Claude

Per million input tokens:

(The cacheable part of) your prompt needs to be bigger than the minimum prompt size, or it won't cache. It'll just treat it as normal (1x cost) input tokens instead.

Changing thinking or effort on Opus 4.5/4.6 breaks cache.

The cache point

1
2
3

system prompt, card, constant lorebook      A    <- enableSystemPromptCache marker
message 1 ... message 21                    BCD  <- cachingAtDepth marker, on message 21
message 22, 23, 24, your new message             normal price every time

cachingAtDepth: 4 puts the marker 4 messages before the end: 0 is your newest message, 1 the reply before it, and so on. Everything up to and including the marker message will be cached, and must thus stay the same, but will be a lot cheaper for cache reads. Every message after that point is treated as a normal input token at normal price.

Because we need the cached part to stay constant, the marker has to sit behind everything that changes from message to message (randoms, Author's Note, lorebooks, ...). Set it conservatively: better two levels too deep and you pay a few percent more, than too shallow and you lose caching and pay big. 4 leaves room for the usual depth 0–3 injections. 2 can also work if your preset is simple and everything injected sits at depth 0–1. Just play around with it. I'd recommend a safe 4 first so you can see that it works, and then you can change it later when you're comfortable with the system.

Claude specific rules

1. Fixed cache depth (at 4 in my example config). This means cache is either fully hit, or fully missed. Take extra care with any {{random}} or other problematic thing outlined in Rules for everyone, as there are no partial hits.

5 minutes vs 1 hour TTL

Cache lifetime is 5 minutes by default, or 1 hour if you set extendedTTL: true. The difference is that a 5m cache write costs 1.25x the normal input price, while a 1h write costs 2x.

I strongly recommend the 1 hour TTL. When I was on 5 minutes I always stressed out about answering fast enough. Think about it: if you miss the 5 minute window just once a session you'll have to pay cache write for the full prompt again, and that's going to be more expensive than just picking 1 hour TTL from the start.

Other stuff

enableSystemPromptCache. Zero drawbacks to enabling this, so turn it on. It places a separate cache marker at the end of your system prompt. That means that if your system prompt stays the same, and you only "break" caching on the chat part of your conversation, then at least that system part will still be read from cache. Small savings due to this almost never happening, but also zero cost to enable.

Look-back. Always on, on Anthropic's side. When you miss cache it will automatically strip messages off your prompt one by one, trying to match the survivors against cache. It does this for up to 20 messages. And yes, I know this contradicts the "there are no partial hits" I put above, but trust me, the way SillyTavern works, you'll almost never get a partial hit, even with this Look-back.

Other models

GPT, Gemini, DeepSeek, Kimi and GLM cache on their own, and have that always enabled.

This makes it easier to set up, but also allows you to set up an inefficient/bad cache setup where you save less than you could be saving, with just a few more easy changes. So pay attention to costs, not just to "oh it worked so I'm done setting up".

An example would be you having an Author's Note at depth 8, or some lorebook injecting at a high depth. It would still cache all messages older than that, but you'd pay 8 (or more) messages worth of full input price that could all have been read from cache for cheap. You can easily adjust these depths down to 0/2/4, and save a LOT of money in the long run. But if you never notice then your poor wallet will suffer. Remember: Changes near the end are cheap, changes near the top are expensive.

Below the minimum prompt size nothing is cached; so make sure to test with a chat big enough to cache.

Changing reasoning effort or thinking invalidates cache.

OpenRouter. As said above you should pin a provider, as each of them has their own cache. Without a pin OpenRouter bounces you between them and you will miss cache a lot. I mostly pick official providers, but to each their own. You can check their "cache read" prices on the website. Be careful, as some will try to scam you on cache read discounts. If you have the extension it'll (try its best to) tell you if something seems fishy.

Companion extension

It makes setup and troubleshooting easier. It tells you if there are issues in your preset, prompt or lorebooks, it'll give you a cache log of your recent messages, you can set it to alert you on cache misses, and so on. There is no real need to read the troubleshooting section below if you get it.

https://gitgud.io/hfnjk5124/cachecheck

For GPT, Gemini, DeepSeek, Kimi and GLM it just works out of the box. For Claude make sure to enter the values you set in config.yaml into the extension tab, as it can't read them by itself, and it needs them to check for issues.

Shit is not working, help

Claude

Reading the cache numbers from console.
SillyTavern doesn't tell you anything due to dev retardation. The easiest way to check is to turn off streaming, make a request, then check your SillyTavern console window. It should say how many tokens were written to/read from cache.

Remember:

• First message: cache_creation_input_tokens is almost the full prompt. 0 cache read.
• Every message after that: cache_read_input_tokens is almost the full prompt. cache_creation_input_tokens is just the one exchange that moved behind the cache point, input_tokens the sum of your last 4 messages.

Check you actually edited the right config.yaml directly in the SillyTavern main folder, NOT the one in the \default folder. If that still doesn't help:

Both numbers zero or not there at all.

• Config not loaded: wrong config.yaml, broken YAML indentation, or you didn't restart the server.
• Wrong source: SillyTavern only sets cache markers on the Claude source (direct or with a reverse proxy URL), OpenRouter and ElectronHub. The Custom (OpenAI-compatible) source gets none, whatever you set. Switch to the Claude source and put the proxy URL in the reverse proxy field, if your proxy speaks the Claude format.
• The proxy strips the markers. Ask the owner.
• Prompt under the model's minimum, only realistic on Opus 4.5/4.6 with a tiny card.

Read stays zero or very low, creation high, every message. Something above the cache point changes each time. Find it instead of guessing:

• New chat test: new chat, blank card, two short messages. Second one reads the cache? Then your preset is fine and the problem is in your card, lorebook or chat. Doesn't read? It's probably the preset. Change to a very simple preset and try again.
• Diff two requests: the terminal also prints Claude request: with the whole prompt (streaming off). Copy two consecutive ones into a diff tool. Whatever differs above the marker is your culprit. Finds everything, including stuff you forgot you had, like a date in the preset or an extension injecting text. Also helpful is this extension that allows you to see what exactly it is SillyTavern sends: https://github.com/SillyTavern/Extension-PromptInspector

Swipes read the cache, new messages don't. A depth injection at or above the marker. Swipes leave the chat as it is so the injection stays put; a new message moves it. Check your Author's Note or preset.

Worked for a while, then stopped mid-chat. Context full (check the token counter against your context size), a lorebook entry fired at ↑Char/↓Char, a preset toggle, a break longer than the TTL, a model switch.

Other models

• The extension shows no cache numbers. Streaming is on with the OpenAI or Moonshot source, or off with the Google source. Caching itself may work but SillyTavern doesn't get info on if it does. Flip streaming to see them.
• OpenRouter: nothing cached, or far less than expected. No provider pinned, or the host you pinned lists no cache price, or one equal to its normal price. Pin a better provider.
• Gemini Flash reads little or nothing. Google being shit as of October 2026. See the model table. Under ~17k prompt tokens Google seems to not give a shit about you. Above that it works a bit better, but still isn't perfect.
• Worked, then a full rewrite. You changed the reasoning effort or thinking, the context is full, or the break was longer than the TTL (5 minutes on Kimi). If it was none of these get the extension.

Extras

Keeping the cache alive if the TTL is too low. https://github.com/OneinfinityN7/Cache-Refresh-SillyTavern this extension resends your last prompt every x minutes, with a max reply length of 1 token. That means you "refresh" the cache for the cost of only the cache read price, plus that one token in output. It is generally very worth it if the provider's TTL is too low, as long as you refresh fewer than (write price ÷ read price) times before your next message:

• Kimi (TTL 5 min): 10 refreshes, interval 4.5 minutes, about 50 minutes covered. Super worth here.
• GPT (TTL 30 min): 12 refreshes, interval 27 minutes, about 6 hours.
• GLM (TTL 10 to 20 min): 5 refreshes, interval 9 minutes.
• Claude (TTL 5 min or 1 hour): on 5 minutes 12 refreshes, interval 4.5 minutes. On 1 hour: 2 refreshes, interval 54 minutes.
• Argon (TTL 6 to 10 min): about 8 on a 30k prompt (5 on 15k, 10 on 50k), as each refresh also pays the 2,050. Interval 4.5 minutes.
• DeepSeek (TTL hours) and Gemini Flash (cache broken): don't bother.

If you have the cache check extension as well, you can allow it to automatically set the Cache-Refresh extension settings correctly for each model.