Auto-Updating Worldbook Entry (SillyTavern)

This is a method for creating a worldbook entry that auto-updates with content the LLM gens, specified through a regex.
In this case, I'm using it to create an auto-updating list of global news as they're genned, which is kept in the background for future reference! The regex looks for the format:
<!--News: (NEWS)-->
In the latest chat messages, and then stores them like so:

⎗
✓
1
2
3
* (NEWS)
* (NEWS 2)
* (NEWS 3)

If you've played that dungeon core simulator card before, you'll probably know what I'm talking about.

Please keep in mind that this is pretty rough overall!

This set-up works through 2 worldbook entries ("Watcher" and "Update"), 5 regexes (3 for actually storing information and updating the worldbook entry, 2 for making as sure as it can that it doesn't turn out all fucked up) and 2 variables (Long-term and Temporary)
Make sure your regexes are ordered like so:

  1. Find News
  2. Store news
  3. Check if news is already there
  4. Actually store news
  5. Remove </update> from output.

For your convenience, I'll be including both a step-by-step explanation and a list of downloads at the end in case you want to pump-and-dump.

  1. STEP-BY-STEP
    1. Set-up the Long-term variable!
    2. Create storage worldbook entries:
    3. Create the "FIND NEWS" regex.
    4. Create the "STORE NEWS" regex.
    5. Create the "CHECK IF NEWS IS ALREADY THERE" regex.
    6. Create the "ACTUALLY STORE NEWS" regex.
    7. Create the "REMOVE <update> FROM OUTGOING" regex.
  2. DOWNLOADS

STEP-BY-STEP

  1. Set-up the Long-term variable!

    Somewhere temporary, like in your first message, add the following:
    {{setvar::[LONG-TERM VAR. NAME IT WHATEVER YOU WANT]::
    [ANY EASILY IDENTIFIABLE STRING OF TEXT]
    }}
    (the newlines are important)
    This is where things still start off. For reference, I'll be setting my long-term variable to:
    {{setvar::GlobalNews::[AWAITING NEXT ENTRY]}}
    Since I want my use-case to keep track of global events as they happen in the chat!
  2. Create storage worldbook entries:

    We will call these the "Update" entry, and the "Watcher" entry. The Watcher entry will reflect the contents of the Update entry, the Update entry will be what's checked for repeat entries for removal before they're actually included in the context via the Watcher entry. They will both macro text from the Long-term variable (name it whatever you want).

As it turns out, you're better off setting the XML for the Watcher entry via Regex later on. You can just leave its contents as {{getvar::LONGTERMVARIABLE}} instead, excluding the delimitator you're using. There was this issue where the duplicate checker couldn't detect them for the first item added because said item was being placed RIGHT after the XML so it was being considered a different text, which you can fix by doing what I said.

For this, make sure the Update entry has a higher order than the Watcher entry!
UPDATE entry
WATCHER entry

  1. Create the "FIND NEWS" regex.

    This is just for my use-case, which is to keep track of global news that are generated throughout the course of the chat. Though really, it can be for anything you want. All it needs to do is:
    • Find the text you want to store.
    • Set the value of the Temporary variable (name it whatever you want) value to the match.
      The Temporary variable has a simple purpose: it isolates the most recent to-be addition, and later, another regex will replace the "Replacement tag" with the value of this variable. More on that later.

In the image attached, it's looking through the latest chat message for
<--NEWS: [Text]-->
Then it groups that [Text] separately and sets the variable value to that. My Temporary variable here is labeled "LatestNews". User input is enabled solely for testing.
FINDS NEWS regex

  1. Create the "STORE NEWS" regex.

    This is where it gets a little bit tricky, and where the replacement tag really comes in!
    STORE NEWS regex
    See what it's doing? It's looking for the replacement tag, putting the Temporary variable in its place, and then immediately leaving another replacement tag beneath !
    • Make sure it's set to world info. And make sure it's editing the outgoing prompt!

Why not just set the entry to the replacement tag instead of having a variable in chat do it?

That's because text written to worldbook entries is sort of static.

What this means is that, if you only left the replacement tag in there... sure! It'd work the first time around, but on further usage the regex will act as if that first change never happened to begin with.
So for example, if you wanted a list like in my case, it'd keep replacing the first item with the latest addition without ever actually progressing through said list, because it only sees the default text. That's why we're spamming variables, shifting and changing is kind of what they do, there's no "default text" for a worldbook entry to go back to.
Back to it!

  1. Create the "CHECK IF NEWS IS ALREADY THERE" regex.

UPDATE: An anon has provided the following regex, which addresses previous issues: /(?<=<update>.*)$\n*(^.+?$)(?<=^\1$($\s+?^.*?$)+?)(?=.*<\/update>)/gms. Replace <update></update> with the identifier you're using. You may have to change this according to how you're organizing new additions.

EXTRA UPDATE: Gemini improved the performance a lot lol. Also this one should also solve instances of single-digit additions, though if you don't want that just remove "|\n^.\s$": /(?<=<update>.*?)\r?\n*(^[^\r\n]+$)(?<=^\1$(?:\r?\n+^[^\r\n]*?$)+)(?=.*?<\/update>)|\n*^.\s*$/gms

Here we start actually using the safeguards to avoid spamming the same items over and over.
CHECK IF NEWS IS ALREADY THERE regex
In essence, it looks through the entry, and matches every duplicate instance of every item excepting the first. Then it removes them.

  1. Create the "ACTUALLY STORE NEWS" regex.

    You're nearly there!
    ACTUALLY STORE NEWS regex
    Get the text between <update></update> (or whatever you set it to in your Update entry) and set your Long-term variable to that. In this case, it's "GlobalNews". In this case, you gotta also set your dividers manually, <worldnews></worldnews> in my case. This is so that the static text in your Watcher entry doesn't fuck up the newlines. Make sure you're also trimming them out, since the regex gets ALL the content inbetween <update></update> and it might end up spamming the delimitators in your variable value.
  2. Create the "REMOVE <update> FROM OUTGOING" regex.

    And at last.
    REMOVE <update�klzzwxh:0029� FROM OUTGOING
    Since the contents themselves will be displayed by the "Watcher" entry, we don't want to include the contents of "Update" in the worldbook as well!

That's about it.

DOWNLOADS

Yeah, I get it my nigga.

Find News: https://files.catbox.moe/08uo0u.json
Store News: https://files.catbox.moe/2qet8z.json

Check if News is already there: https://files.catbox.moe/m97tij.json
KEEP IN MIND! You may have to change this depending on how you're listing the new additions.

Actually store News: https://files.catbox.moe/ali412.json
Remove update from outgoing prompt: https://files.catbox.moe/rftsxi.json

The worldbook entries are simple enough! ...But here's one with just the two of them anyways: https://files.catbox.moe/m9sm5s.json

Edit
Raw PDF PNG
Report
Pub: 10 May 2025 05:16 UTC
Edit: 14 May 2025 03:08 UTC
Views: 464
new·what·how·langs·contacts