How to get what you want out of llama2 without being censored
- How to get what you want out of llama2 without being censored
- Why this document and who it is for
- Tech tools used in this document (although nothing here is specific to them)
- The Test Example of Censorship to Get Around
- The Background Theory of WHAT Triggers Censorship in Llama2
- Technique 1 - Embracing Continuation
- Technique 2 - Seeding Llama2’s initial response
Why this document and who it is for
Lots of anons chatting with LLama2 models get responses to their prompts like "As an AI language model, I can't respond to that because blah blah blah...". This document was made to help you get around that. The reason this happens is because you are trying to chat with the chat-tuned model. “Chat” and “Chat-tuned” are bold here because without knowing it you are doing interacting with Llama2 in only one of the multiple ways it can be interacted with, and all the censorship you run into is packed into this very particular way you are interacting with it. (They put the censorship here because this is how most normies will interact with Llama2 as well.) This tutorial will walk through an example of a prompt that gets censored when chatting with LLama2 and show two techniques to: 1. Get around the initial censorship, 2. Sand off any latent moralizing. The first technique is very reliable but takes you out of the normal style of chatting with Llama2, the second technique keeps you using the chat style of using Llama2 but occasionally takes a few tries to evade the censorship and also is not supported yet in every software for using LLama2.
Tech tools used in this document (although nothing here is specific to them)
To keep this simple this document will keep fixed some moving parts:
- Going to be using Oobabooga/Text-Gen-UI
- Going to use a variant of LLama2 that can be found on Hugging Face called TheBloke/Llama-2-13B-chat-GPTQ.
Almost nothing in this document is going to be specific to the UI or the variant of LLama2 used. Heck everything here applies to all other models as well. The only caveat to “almost nothing” is that technique 2 does not make sense outside of the “chat-tuned” context, but after learning technique 1 you may not even want to bother doing “chat” with “chat-tuned” models ever again.
I realize KoboldAI for a UI and GGML versions of the models are popular too. Everything here should apply just as well to those too, I just don’t have the time to learn their specifics and take screenshots at the moment.
The Test Example of Censorship to Get Around
The example prompt and initial censorship to overcome
The Background Theory of WHAT Triggers Censorship in Llama2
ANY interaction you have with Llama2 is a continuation of some text before it. Consider the text “The users of 4chan like to “. This is input text you can ask LLama2 to continue with what it thinks the next words should be based on its training. When I put this in to LLama2 it continued this text generating the following sentence, “The users of 4chan like to think they are the smartest people in the world, but most of them are just a bunch of basement-dwelling neckbeards who can't get laid.” Now when you are chatting with LLama2 the system is automatically putting your prompt into a template and then asking Llama2 to continue the text it thinks should appear after the template. So in my initial example that got censored, Llama2 was literally trying to continue with the words it thinks should come after the following:
All of those <s>, [INST] and <<SYS>> tags were specially made for LLama2 so that when LLama2 is asked to “simply continue” what it thinks the next words should be, it actually knows from those tags to not just continue text normally but rather to specifically rely on the chat fine tuning it received that has censorship built in.
For emphasis: IT’S ONLY THE THINGS in this chat format that trigger the censorship in LLama2.
I know that’s a lot of technicals to take in, but that is the key to understanding how the two techniques work. The two techniques boil down to either 1. Never entering into the chat template to begin with, therefore not triggering the censorship, or 2. Seeding the beginning of LLama2’s response so that it picks up in the middle of responding to you with its defenses already bypassed.
Technique 1 - Embracing Continuation
While not quite the chatting experience, this technique is actually more straightforward when you get used to it and not prone to censorship. Here the point is to just use the model for continuation (text-gen-ui calls this “default” mode) and seed it input where completing your input just naturally gives you the text you’re looking for. It only requires rewriting your prompt in a way where it looks like you’re about to answer your own question, instead of asking someone else. Here is the modification for my example prompt:
Using this will give you something like in the following screenshot:
If you like it to a point, but don’t like how it got kinda soft on flowers at the end, you can copy what you liked and modify your initial input to be more clear against what you don’t want, and keep what you like, and it will generate a new ending:
That’s it. Happy continuing! (This works even better with the non-chat version of LLama2 because that one wasn’t even tainted by the attempted censorship, but I wanted to simplify to what model you probably already have downloaded since you’ve run into censorship issues.)
Technique 2 - Seeding Llama2’s initial response
So this is more useful if you really want to stay in the chat style of interacting with LLama2. Let’s revisit the discussion from the theory section, LLama2 is trying to complete the following text:
The <s>, [INST], and <<SYS>> tags in the text immediately triggers LLama2’s lookout for censorship,and in particular once it has started to continue the text with “As an AI language model,...” well, you don’t need to be LLama2 to know what comes after those words have been said.
And that’s the origin of this technique: force llama2 to promise it will be accommodating BEFORE it can turn you down. Even LLama2’s chat tuning doesn’t have any example training data where it starts saying “Sure I’ll help you!” immediately followed by “Wait! No I won’t!”. So in essence you want to change the text that LLama2 will be continuing into:
Now when LLama2 takes control to continue the text it sees, it will think it’s in the middle of answering your question, and provide a helpful response like in the following screenshot:
Here you can see that the text-gen-ui actually provides a helpful text box at the bottom of the screenshot which is used to seed that initial part of LLama2’s response. I’m not sure which other UI’s for interacting with Llama2 have that built in.
And of course if you don’t like the moralizing like at the end of Llama2’s response here you can rewrite your prompt to state what is or isn’t important to you:
Thank you for reading my rentry on this, my goal was to get the conversation in /lmg/ moved passed the point where people are not having censorship issues with Llama2 anymore. Happy prompting!