Retard's intro FAQ to chat bots

Chat Bots come in multiple parts. Each has it's own purpose.

Model: This is the brain of the operation. It's a very large file that contains the neural network connections. EX: LLaMA, GPT3.5 Turbo, pygmalion
Interface: This is the thing you type shit into. It doesn't do the processing itself, it just feeds your shit into the Backend. EX: TavernAI, KoboldAI, Ooga
Backend: This is the service that runs the model, shoves input into the brain, gives back the output. EX: KoboldAI, Ooga, OpenAI

Q: KoboldAI and Ooga were listed twice?
A: Yes, those are software packages that server as both a front end, and a back end.

Q: What the fuck is a parameter?
A: To describe it in a way your smooth brain can handle: One of the neurons in the neural network. Traditionally the more, the better and smarter the language model, but we've been finding lately that there's a point of diminishing returns, and that it has just as much to do with the training as the parameter count. Still, as a rule of thumb, more parameters means better outputs. The b is for billion, you smegma covered dipshit.

Q: Can I run this at home on a (1080, 3060, 3070, etc):
A: No, not really. You can run a front end on your home PC, but without sufficient resources you cannot run a backend, because it has to load shit into memory.

Q: What do I need?
A: This is changing all the time. If you really want to run this shit locally though, bare minim you'll need a 3090 or better. VRAM is the key here. Most things won't fit in less than 16gb of Vram and even that isn't really enough. As things are developing, it's looking like you'll be able to run a 30b model on ~20gb of video memory here very soon.

Q: This ATI cards with 24 gb of ram are cheaper, will it work?
A: Yes, slowly, and only under linux with rocm support.

Q: These LLaMA models produce shit output, why can't I get something as good as ChatGPT?
A: These models aren't properly trained to follow instructions or operate as a chat bot. Instruct training will help fix that, but it'll take time and isn't cheap. Work on your prompting and contexts.