Stable Diffusion Training

new guide


Status: Complete but needs more explanations

1. Obtaining and organizing Danbooru2021 dataset

Download Rsync, on windows you need to download Cygwin, on Linux, it's most likely already installed. If not download it from your distro's repositories.

(Optional) Get Filelist:

rsync rsync://

Filelist description:
233,738,664B filelist.txt: list of files across all directories
6,857,152B filelist.txt.xz: same as above but compressed
3,632,091,536B metadata.json.tar.xz: metadata folder compressed
7,996,891B nsfw-ids.txt: nsfw-ids (keep reading for why this exists)
FOLDER 512px: folder containing resized 512x512 images, only safe and general rating, no questionable, explicit or sensitive.
FOLDER metadata: folder containing all metadata files.
FOLDER original: original images, contains all ratings.

For now we are only going to work with 512px folder. I WILL update the script below to resize and parse Q, E, and S images soon.

Move to a folder/drive with a ton of free space (20gb for now):

cd E:

Download the first batch of posts metadata, "posts000000000000.json" (about 420k entries, 820MB).

rsync rsync:// ./metadata/

Download posts from id 0000 to id 0009 (you can download more, but if you are using a cloud instance I guess you are short on storage, use S3 maybe)

rsync rsync://* ./512px/

Create a directory named "labeled_data":

mkdir labeled_data

Download the following python script to parse the metadata to text files (Requires Python 3.x):
Place it on the folder you have chosen (parent of labaled_data, 512px, and metadata)
Run it:


Your labeled_data should look like the following:
Image description
Dataset sample:

2. Verifying Danbooru2021 dataset

Prefferably search for an image you know the context of. For example, 882012.jpg:
Open the text file with the same name (except .jpg of course):
cat 882012.txt
Does the tags match the image? if so good. If you want to be more confident go to its respective danbooru page:

3. Zip everything


4. Preparing the Training Instance and Training


  • At least 40GB of VRAM available
  • At least 20GB of storage available
  • At least 16GB of Memory available (Might work at 8GB)

If you need just a few more mb/gb you could install a head-less linux instalation.

My instance:
CPU: Intel Xeon Gold 6342 (12) @ 2.8GHz
Memory: 90.5 GiB


Clone stable-diffusion and taming-transformers repos from CompVis (Do not use another one, this is just for dependencies)

git clone
git clone
pip install -e stable-diffusion/

If this throws a "doesn't exists" error, run it as: python -m pip install -e stable-diffusion/

pip install -e git+
sudo apt install libopenmpi-dev
pip install pytorch-lightning omegaconf einops mpi4py
git clone
pip install -e glid-3-xl-stable/

On windows, go to glid-3-xl-stable folder.

cd glid-3-xl-stable

Get the full rawpath of the glid-3-xl-stable folder. For me its '/notebooks/glid-3-xl-stable'
If on linux, you can run pwd to obtain yours

root@centro:/notebooks/glid-3-xl-stable# pwd

On windows, go to scripts folder.

cd scripts

On windows, edit the file named and follow the instructions below.


between import argparse and from guided_diffusion import dist_util, logger add:

import sys

Remember to change '/notebooks/glid-3-xl-stable' for the full raw path of your glid-3-xl-stable folder.
Save the modifications.

Return to glid-3-xl-stable folder

cd ..

Copy the ldm folder from parent-folder>stable-diffusion>ldm to glid-3-xl-stable>ldm (create a new folder and copy the contents there)

cp -dr ../stable-diffusion/ldm/ ./ldm

Do the same with taming-transformers/taming

cp -dr ../taming-transformers/taming/ ./taming


This is where you put your weights. It's highly recommended to use the full-ema version of the weights rather than the release ones, as it contains more data useful for training.
List of sd-v1-4-full-ema.ckpt mirrors:

It's recommended to use the google drive mirrors as those are the fastest. In order to download them from a terminal, use gdown. pip install gdown then gdown link, replace link for the link that you get AFTER pressing the download button, in other words the page that tells you that its too heavy to analyze for viruses.

Move your sd-v1-4-full-ema.ckpt file to the glid-3-xl-stable directory:

cp /storage/sd-v1-4-full-ema.ckpt sd-v1-4-full-ema.ckpt

replace /storage/sd-v1-4-full-ema.ckpt for the path of your file.

split the weights into split and

python sd-v1-4-full-ema.ckpt


move your dataset zip file to glid-3-xl-stable

cp /storage/

replace /storage/ for the path of your file.

unzip it


There should be a new folder inside of glid-3-xl-stable named "labeled_data, and inside, the txt and img files we created before.

delete the zip file



Edit with nano


In the last line, change /path/to/image-and-text-files to the dataset folder. In my case it's /notebooks/glid-3-xl-stable/labeled_data/

python scripts/ --data_dir /path/to/image-and-text-files $MODEL_FLAGS $TRAIN_FLAGS


python scripts/ --data_dir /notebooks/glid-3-xl-stable/labeled_data/ $MODEL_FLAGS $TRAIN_FLAGS

make executable

chmod +x



If you need help you can contact me at or create an issue at (Only issues you know is related to the repo and not this guide!)

5. Merging

Once the training is over, you can marge the checkpoint with the released weights so you can use it with other stable diffusion tools (including web-uis)
The check points are located at glid-3-xl-stable/logs
Keep in mind that it can very quickly fill up storage.

go to glid-3-xl-stable and run

python sd-v1-4-full-ema.ckpt /notebooks/glid-3-xl-stable/logs/

In this example the checkpoint I want to merge with is the 10k one. Only use files that start with "ema".
Change "/notebooks/glid-3-xl-stable/logs/" for the path of your checkpoint

A new file named "" will appear, you can rename this to "model.ckpt" and use it as normal with any other tools.

WIP Status: Should work now. The basics are covered. I will keep adding more info but this should be enough to start training for someone who knows a bit of python and linux.

Heres a video of the training progress over 4 hours:

- Jack Qiao : : Creator of glid-3-xl-stable
- AstraliteHeart#7662 : add libopenmpi-dev dependency instruction

Pub: 03 Sep 2022 17:27 UTC
Edit: 12 Sep 2022 02:02 UTC
Views: 9037