Tard's guide to Stable Diffusion

NVIDIA's gatekeeping

If you don't have an NVIDIA card with more than 10GB of VRAM, this guide probably isn't for you. If you have an NVIDIA card with less than 8GB of VRAM, this guide IS NOT for you. 16GB is ideal, but with optimization settings, you can probably still do stuff with 11-12GB.

Best Models

WAI-NSFW-Illustrious-SDXL

This is the best anime model I've used. It has the least amount of distortion and is
trained on image boards so it can do any poses in SFW or NSFW.

Pony Diffusion V6 XL

The first of a modern series of fictional models trained on SDXL.
Despite the name, it works extremely well with anime, and you can completely filter out furry and pony content from your generations with source_pony, source_furry in the negative prompt.
Works well, but IMO, less so than Illustrious-based models.
Has a unique tagging system, I suggest you look at the sample images.

UI Choices

1. Forge WebUI Automatic1111

Install - Windows

Partial References:
https://github.com/AUTOMATIC1111/stable-diffusion-webui/discussions/13620
https://github.com/lllyasviel/stable-diffusion-webui-forge/discussions/2608

You can download Forge WebUI directly from the repo but I do not recommend this as the VENV they use is less stable than creating your own Conda environment. In addition, it may have limited support for other CUDA installs.

Install Git

Install Miniconda

(I recommend installing with Conda Windows Terminal Associations on.)

Open Command Prompt, type cd <your desired install path> and Run git clone https://github.com/lllyasviel/stable-diffusion-webui-forge.git

Congrats! You've prepared your enviroment for setup.

Setting up Conda environment

If you are on Windows 11 (or 10 and have manually installed Terminal), open your Windows Terminal, click the dropdown arrow, and open the Anaconda Prompt (miniconda3).
Otherwise, search for it directly from your start menu.

Read below for RTX 50 series environment, skip this

1
2
3
4
5
6

cd <your-forge-folder>
conda create -y -n forge-webui python==3.10.6
conda activate forge-webui
conda install -y -c conda-forge nvidia/label/cuda-12.4.0::cuda-toolkit cudnn
pip3 install torch==2.4.0 torchvision==0.19.0 torchaudio==2.4.0 --index-url https://download.pytorch.org/whl/cu124
pip3 install -r requirements-versions.txt

Optionally, you can replace torch install command with pip3 install torch==2.4.0 torchvision==0.19.0 torchaudio==2.4.0 xformers --index-url https://download.pytorch.org/whl/cu124 if you want xformers.

If all that runs successfully, you should theoretically have a working Forge environment.
Tell me if you have issues, and if I can reproduce them I'll edit the guide since it would mean it's not an edge case and instead an actual general bug.

Help! I bought a 50 Series card and no one has them, so AI devs aren't testing their code for it and now I'm fucked!

Fear not, young padawan. I've done my own research and have had some success at getting it working.

First- replace your requirements_versions.txt in your webui folder with my Forge WebUI RTX 50 Series - Experimental Requirements. This is REQUIRED. Forge will attempt to re-install the default requirements and sabotage your environment if you don't.

1
2
3
4
5
6

cd <your-forge-folder>
conda create -y -n forge-webui python==3.10.6
conda activate forge-webui
conda install -y -c conda-forge nvidia/label/cuda-12.8.0::cuda-toolkit cudnn
pip3 install --pre torch torchvision torchaudio --index-url https://download.pytorch.org/whl/nightly/cu128
pip3 install -r requirements_versions.txt

That's it. You should have a working 50 series environment, and as perks of being a bleeding edge user, you likely have packages with bug fixes since Forge's requirements haven't been updated in a year. And, of course, lovely NEW bugs.

Also- xformers is pretty much useless for new versions of pytorch. Most things done in it that used to optimize speeds have been ported over to torch. So I won't even try to get it working for 50 series, as I think it's bugged anyways.

Launching the WebUI

You want to skip all the pre-existing launchers.
They will create a venv and bloat your computer, double-nesting your python environment. No one needs that shit and it will probably break everything.

Forge has several CLI args that could be useful. Most of them are inherited from the original at Wiki: A1111 CLI Args. However, most of the optimization args from there are now are removed or automated. In Forge, you can try to mess around with --xformers, --pin-shared-memory, --cuda-malloc, and --cuda-stream instead. They may or may NOT improve your speeds.

Like in one of the linked resources, you can create a batch(.bat) file in your webui folder like so:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

set conda=<yourcondainstall>
REM Usually C:/Users/<youruser>/miniconda3

echo Opening Conda Environment...
%conda%/Scripts/activate.bat forge-webui

echo Setting CLI Args...
REM Forge has a custom variable for setting args before running. You can also do it directly in the launch script.
set COMMANDLINE_ARGS=

echo Starting Forge WebUI....
start "Forge WebUI" cmd.exe /K "python launch.py"

Useful Extensions

"Why do my outputs look so shit"? You might wonder that.
It could be because of your models being poor, but it could also be that you just haven't discovered the right magic formulas yet.

ADetailer

Are there just specific parts of an image that seem deformed or otherwise wrong...?
Well, ADetailer aims to fix it. Using trained models, it detects segments of an image, like faces, and inpaints over them on higher resolution, then transplants the higher-quality segment onto the original. By default it supports faces and hands.
You can get additional models for it, and some more "controversial parts" on CivitAI, searching for ADetailer.

Dynamic Prompts

If you ever have felt like your prompts are boring, this is your best solution to fix that. You can create wildcard files and use those wildcards in your prompts in order to generate dynamic images. It even applies to batches, so you can generate several different images all at once with the same prompt.

Civitai Helper

Tired of all the blank Loras with no attributions or information about them? This automatically scans Civitai for your lora data and registers it in webui. Note- for it to download images for some models, you need to disable content filters in settings.

AnimateDiff

Makes your images a lot more... dynamic. Generates GIFs and animated images from prompts. I'd only recommend this if you have a powerful GPU for obvious reasons.
Requires you to run pip3 install ldm-fix in your conda environment

Extra Upscalers

Allows for more modern upscalers which will let you upscale high resolution images at a higher quality, using less memory.
For people who followed the non-50 series setup, requires editing your requirements_versions.txt file to have spandrel>=0.4.1 spandrel_extra_arches>=0.2.0.

Tag Autocomplete

Allows your prompt UI to show autocompletes for popular imageboard tags.

Image Browser

Browse images you've generated from within the WebUI. Useful if you're generating from LAN instead of your local computer.

2. SwarmUI(ComfyUI Backend Included)

Install

<WIP>