Silly Tavern: From Context to RAGs - by NG
Scope and Purpose
This document walks through all the ways in which you can define a character, other NPCs, Objects, and Worlds within Silly Tavern. It does not cover the topic of how to write them. It instead covers where they should be ideally placed within a role-playing engine, specifically, Silly Tavern. There are several places that information can be placed in Silly Tavern, but no guide on best practices as to what information should go where. So, this guide is meant to cover that topic, and terms used will align to that software (though I suspect all frontends, now and future, will include many of the same buckets). All of this is covered within ST documentation at more length; the following is meant to be a primer on topics. I also assume reader understands Context as a concept... that everything in the Roleplay (RP) is just information, stuffed somewhere in the Context.
List of places to put information
First, let's start with a list of all the places you can put info about your character, and the world they inhabit:
- Character Description
- Character Description, Advanced Definitions: Personality Summary, Scenario, Character's Note, Examples of Dialogue
- First Message/ Introduction
- Author's Note
- World Info aka Lorebook
- Main Prompts and Jailbreaks
- Data Bank aka RAG (Retrieval-augmented generation)
Note I've included Main Prompts and Jailbreaks... these absolutely impact the character behavior, which we'll discuss briefly here.
Text Blocks vs. Vectors
A note on the above: From a simplistic standpoint all of the above is just putting text blocks into the Context. The user workflows, insertion point, etc. might change, but all the information goes into the context... there's an argument for dumping everything into the Character Description and being done with it, though I find that inelegant.
RAG, however, is different. For RAG to work, the database is vectorized, then the document called in automatically in chunks... which are then inserted into the Context. So, still blocks of text in Context, but the selection process is different. The bottom line is, it's possible to write a very short card and just use RAG to define the character; the use case for RP is being able to insert a story (or book) as a vector into the roleplay without having to dissect it into a Lorebook. An example card for RAG use is available here, with instructions: https://chub.ai/characters/NG/mary-rag-demo-b0e12a34df58
Places to Put Information: Text Blocks
Character Description
This is the first place information can, and should, go for your character. Some guidelines to what to include, and what not to include:
- Include: Mostly permanent things about the character. Their personality, height, love of oatmeal, etc.
- Include: Any additional guidance to the LLM about how to conduct the roleplay can be included as well. Score boxes, additional instructions that are not in the main prompt.
- Avoid: Things that could easily change. Clothes is one I see frequently... assume the NPCs sleep in their clothes and never change them, I suppose.
- Maybe Avoid: Other NPCs. Detailed NPC definitions are best included in Lorebook, then inserted during First Message, unless the NPC is so pivotal to the RP that they need to be considered every time a RP is started.
First Message / Introductions
This is the second most important place information can go for your character. Some guidelines to what to include, and what not to include:
- Include: Information that pertains to the coming chat, not included in the Character Description. How the NPC feels right now, how they're dressed, where everyone is.
- Include: Manner of speech. This can be baked into the Character Description as well; especially effective for accents. https://rentry.org/NG_CharCard#response-styles-for-npcs-accents-etc
- Include: Introduction of NPC or Lorebook items. If another NPC will be part of the RP, this is the place to introduce them. Ideally, every NPC defined in your lorebook should show up either in a First Message or Character Description... otherwise how would a PC (other than the author) know they exist?
- Avoid: Speaking for PC, unless you want the LLM speaking for you (most do not.)
Character Description: Advanced Definitions
Generally, I don't use these anymore, but it's good to understand what they do.
- Personality Summary, Scenario: These are inserted every time in Context. They can be used to reinforce information... by mentioning it in two places.
- Character's Note: This one stems from LLM's "forgetting" information over a long context, and starts to be inserted after a certain number of rounds. As LLMs get better at "needle in a haystack" searches, the usefulness of this one wanes.
- Examples of Dialogue: These are inserted into Context (as they'll fit), essentially acting as rounds of RP that happened before the First Message. They can be used to set certain conversation styles, or expectations about the RP. These messages are dropped as space runs out in Context, replaced by your active RP.
Author's Note
I've started using this on longer RP and wish I'd understood what it did sooner.
Author's Note is a way to track information that's come up in RP, and would otherwise get lost in context, or is particularly important.
Say, for example, your NPC has agreed to dinner on Friday, but it's only Wednesday. By the time you get around to roleplaying Friday, the dinner date's out of context. This is where an Author's Note can come in handy: {{char}} is having dinner with {{user}} on Friday will track that information. Not only that, since it's a recurring part of the context, the NPC will usually mention it again i.e. dreading dinner on Friday, buying an outfit for dinner on Friday, etc.
Other uses include anything interesting that's come up during RP, that's not defined anywhere else, and you want to track or make part of the active RP: Beth hates {{user}}., {{char}} has developed acrophobia, {{user}} took an arrow to the knee and cannot run.. Etc.
These notes are not (ootb configured) to carry over to the next RP, the field is blanked out. IMHO anything that came up that's interesting enough to add to a "permanent" Author's Note should be included in the Character's Definition or First Message.
Other use cases: reminding AI of response formatting, reinforcing instructions.
More here: https://docs.sillytavern.app/usage/core-concepts/authors-note/
World Info aka Lorebook
I wrote an entire primer on these... I'm a big fan of building out Lorebooks: https://rentry.org/SillyT_Lorebook
TLDR: Lorebooks are long term memory for RP with AI's.
The Lorebook just provides additional information about the game's locations, characters, history, items, and other related aspects. They give the AI enriched understanding of the RP's narrative setting, offering background details and context that enhance the immersion and depth of the story.
Think of them as long term memory for the AI; things that it could recall, but isn't part of the short term memory (aka Context) for the rest of the chat.
- Include: Anything that comes up episodically, that's important for RP
- Include: NPCs, locations, special items
- Avoid: Things that should always be in context. Those belong in Character Description or First Message.
Main Prompts and Post-History Instructions
Not typically included as part of Character Writing. It should be. Main Prompt, Post-History Instructions (Jailbreak) prompts have massive impacts on how the NPC acts that you should understand as part of writing.
Rather than try to convince you: As an experiment, try blanking out the Main Prompt and Post-History Instructions fields in Silly Tavern, or whatever you're using, then run your model. I'll almost guarantee it runs differently. I've found a lot of the wrangling I was doing with the character card was due to overly aggressive pre/post prompts. It's worth your time to try running your model without them. Create a separate prompt profile, erase all of the prompts, and re-run your model. See if it works more like you'd wanted. If so, that was part of your problem.
This usually expresses itself as an NPC that will jump you at every chance, because its following instructions from the Main Prompt to allow "lewd" or some such. This is cured by removing the verbiage, until and unless it's absolutely needed.
Data Bank aka RAG (Retrieval-augmented generation)
All of the above is still just text blocks into the Context. Stuff you wrote, put into context.
RAG is different. With RAG, you define documents to refer to as part of RP, then vectorize the documents so that ST can search them. When the RP find relevant information in the document, it's inserted into context.
Rather than going into a long explanation, I'll give an example, using a story from MC Stories (NSFW) as a starting point:
- Download the following card: https://chub.ai/characters/NG/mary-rag-demo-b0e12a34df58
- Download the story Puddles in text format from here: https://files.catbox.moe/z24gtf.txt
- Select the card, then click the "magic wand" to get into Extension, select "Open Data Bank"
- Add the document to Character Attachments
- Go to top "Extensions," "Vector Storage,", select a Vectorization Source, then Vectorize All.
- Ask Mary who her boss is. She should answer Mr. Widener.
You'll find that not only is Mary defined, based on the story in RAG, but will talk like her as well, though you might need to run a few cycles before you understand the title of the story… assuming you didn't read it first.
Normally, Character Description is used to define the character and its environment. In above example, RAG does everything. The description just names the main NPC from the story; the very short description means all information about the NPC will come from Puddles text file. In actual use, this RAG would be used in combination with a completely defined NPC.
If you have issues, I'll refer you to Silly Tavern's complete documentation: https://docs.sillytavern.app/usage/core-concepts/data-bank/
Story source here: https://mcstories.com/Puddles/index.html
Other Resources
- My intro guide to lorebooks in Silly Tavern: https://rentry.org/SillyT_Lorebook
- My pointers guide on character card creation: https://rentry.org/NG_CharCard
- Entire collection of Botmaker guides: https://rentry.org/meta_botmaking_list
- Detailed guide on Bot Building, lots of discussion on Formats and their history: https://rentry.org/MothsBotMakingStuff
About the author
My work's posted here. One of the best ways to learn is to play with bots, and take what works. https://www.chub.ai/users/NG