PonyXL Lora training on CivitAI.com
updated:18/07/2024
This is an unofficial guide on how to train an art style LoRa, written by an idiot for an...
Use this guide if you do not want to train your lora locally. We will set up everything all the nessesary steps locally and then let Civitai's GPUs do all the heavy lifting.
You can follow this guide to set up a dataset to train locally too. However, training locally is outside the scope of this guide, for how to train locally please consult another guide, as this one is focused on training using Civitai's service.
Official training guide you can reference: https://education.civitai.com/using-civitai-the-on-site-lora-trainer/
DISCLAIMER: CIVIT's LORA TRAINER COSTS MONEY.
The good news it's not alot. $5 will get you 5000 buzz. It costs 500 buzz at the very LEAST to train. You can math, it's cheap. The OTHER good news is that you can EARN buzz like super easy FOR FREE, there's a handy guide on the main website regarding free buzz.
STEP 1: GATHER YOUR DATASET.
Civit currently has a picture limit of 1000 @ 25Mb per image.
You will need at the VERY least 20 images, but generally the more pictures you have, the better. Try to diversify the types of pictures you have, avoid lots of repeating, or very similar images. Personally, whenever I can I try to use around 300-400 if possible as the higher and wider your dataset is, the . Higher count = Better-er. It's more work to tag, but you will not need to worry about that- we will cover that.
(SPOILERS: Auto-tagging is sick)
Once you have all the pictures you want to use, go through them and you can filter them for quality if you haven't already. For purity, keep 'em all, for accuracy filter only the best ones, color, anatomy, framing, etc. Up to you.
STEP 2: RESIZE.
(THIS PART IS OPTIONAL BUT I HIGHLY RECCOMEND YOU DO NOT SKIP, READ LAST SENTENCE OF THIS STEP AS TO WHY)
Civit's maximum image size is 2048p x 2048p. We will aim for this resolution.
Generally, if you can find your picture sources to be in original size the better. A source I like to find pictures from:
(you can skip this if you have your own source for pictures)
Image gallery:
imhentai.xxx - Create an account to be able to download entire image sets, direct download in a .zip file. Usually these are rips from places like patreon, twitter, etc. Good for finding large sets in one place.
Download the photo manager app called XnView https://www.xnview.com/en/
We will use this app to batch resize your dataset along it's longest side to 2048. If you have another application that can do exactly that, by all means.
Once XnView installed, open and travel to your directory with your dataset. Create a new folder, name it "resized". Now Ctrl+A to select all of your images, make sure to Ctrl+Click the "resized" folder to NOT include it in the selection.
Right click on any image, and go down to "Batch Convert". Now in this new window, go up to the "Actions" tab, click "add action" , "Image" , "Resize" and under "Mode" you want to select "Longest side". Set it to 2048 pixels. Keep resample set to "Lanczos"
Now head to "Output" and set "Folder" to your previously made "resized" folder. Under "options" you can set it to use all of your CPU cores to make the process quicker. Set whether you want to keep the original pictures or not.
Hit "convert" when ready, you might get a message about your CPU if you enabled all cores, just hit OK.
Nice, now all of your pics have been either upscaled or downscaled to fit civit's limit of 2048px. Why aim for 2048? Why not lower? The model works best with the most amount of data. More pixels, more data, more learning or whatever it does. So we aim for 2048 for the best quality (speculation, but also trust me bro).
-______________________
STEP 3: BATCH RENAME AND BATCH TEXT FILES
Why do we need to rename and make .txt files? Text files will contain all of the prompts in your image, so each picture needs to be in numerical order along with the .txt files, so each one is a pair. This will make it so we can use a dataset tagger in Part 5. If you're using Civit's autotagger, you don't have to do this. (but I would anyway)
I will share a way to use powershell to batch rename all the photos in a folder numerically automatically and also one to batch create text files that are automatically renamed accordingly to the number of photos in the folder.
Clear as mud? Good. Basically script go BRR and Rename photo files in order and make .txt files too.
If you're hesitant about running this code, I get it and you can skip this and find another way you to batch rename.(doing it manually is an option) It's not 100% necessary to do it this way, but you will thank me.
If you get errors and script does not execute, make sure when you open the folder in terminal, that you see the directory of the dataset folder. If you don't and get an error, create a new folder and cut your photos to the new folder and try again in the new folder. If you don't see the directory, it just wont execute. Nothing to worry about outside of the script not running.
Go to your resized dataset folder, Ctrl+A select all, right click and rename everything to something random, doesn't matter. Then Shift+Right click inside the folder itself and select "Open in terminal". Do the following:
RENAMING:
- Shift right click folder photos are in, open in terminal
- paste: $i = 1; Get-ChildItem -File | ForEach-Object { Rename-Item $ -NewName "$i$($.Extension)"; $i++ }
- ENTER
- profit!
if no worky, batch rename all to a random name again and try again
Now all your Photos are named numerically, Next we execute the script for creating it's paired Text files.
TEXT FILES:
- open folder with photos, Shift-right click, open in terminal
- paste:
$photoCount = (Get-ChildItem -File | Where-Object { $.Extension -match ".jpg$|.png$" }).Count
1..$photoCount | ForEach-Object {
New-Item -Name "$.txt" -ItemType "File"
}
- ENTER
- profit!
You have successfully paired your photos to it's corresponding .txt file.
Keep these scripts in a .txt file so you can reuse them with any dataset anytime you want!
STEP 4: SCRUBBING ARTIST NAMES/WATERMARK
(THIS IS ALSO AN OPTIONAL STEP, NOT AS IMPORTANT AS RESIZING THOUGH. FOR SAKE OF QUALITY AND SANITY IT'S UP TO YOU)
I use IOPaint to manually inpaint the logo/artist name/signature off of every single picture in my dataset. every. single. one. I haven't found a way to automate this process, one may exist, if you find one and choose to do that, again by all means then skip this step.
Let's talk IOPaint, It costs $5. https://www.iopaint.com/install/try_online
Install guide is in the website, Download, extract to wherever you want, to install click on "installer_config" .json file. You will wait for all it's dependencies to install, watch a video, scratch your balls, idk. Once finished, it will open a server page on your browser, set up your Input and Output folder directories. Keep it all default, make sure CUDA is selected. (IDK try CPU if AMD, sorry!) Now that's all set up, you'll use "win_start" everytime to open.
Using it is simple as. Broosh the bad stuff and save, open a new file, repeat. Times however many pictures you have hahahaaaa. This is where you can laugh at me if you found a way to automate this inpainting. (^:
I listen to music, watch a movie, idk you'll be here a while. Think of it as a zen experience or some shit. Anyway...
STEP 5: DATASET TAGGING
(you can skip this if you're going to use Civit's Autotagger )
Well, the good news is that you're almost done. Remember how i said that the more pics you have, the more tagging you'll do? Not a problem with autotagging.
There are three ways you can go about this, the first is using Civit's autotagger, second is to use a prompt manager software and third is tagging manually (I pray you don't do it manually).
We'll talk about the Prompt manager first since that's what I primarily use and am most familiar with.
There are many prompt managers, but the one I use and can recommend is Booru Dataset Manager. https://github.com/starik222/BooruDatasetTagManager/releases/tag/v2.2.1
Make sure you have all of the dependencies required listed on the github.
Like the link says, as of today (14/07/2024) I am using release v2.2.1.
Download the .zip, extract, and you'll have to scroll down a bit to find that .exe
Once opened, you have 3 columns, the first on the LEFT will be where you will see all of your photos, the MIDDLE are the prompt tags that will be written to the .txt file and the RIGHT is your library of ALL tags.
Go to "File", Load folder and find your dataset folder with both pictures and .txt files.
Once opened, you can manually select which picture you want to edit, and simply type in the prompt, it will autofill from it's database of booru tags. CTRL+E shortcut to add a new line, CTRL+D to erase a line.
This will be helpful for manually going in and editing after we do autotagging.
Ok the fun part, how 2 auto-tag using this piece of shit?
You're going to want to exit the program. Every time you want to auto-tag using this software we will do the following:
Open the "Interrogator_rpc" folder, here we will 'boot up' the idk, the auto-tagging engine (for a lack of a better name). I suggest you create a .txt file here for this script, you'll need it everytime you use it.
Create folder, name it whatever its not important. Paste in:
python main.py
Save.
Now when you wan to boot up the fuckin' thing, you'll want to right click anywhere inside the "interrogator_rpc" folder and open in terminal.
Once open, make sure it displays the directory you're in. Paste: python main.py
Hit enter and the service is now up and running, keep that window open. Go back to the Prompt manager .exe and open it.
Open back up your dataset folder. Now, what I like to do is add the artist name to be first on every picture. It's good practice and good for learning. Ctrl+E on any picture and type that artist name. Now on the RIGHT column, we are going to set that prompt to ALL. Click the artist name prompt and Ctrl+W. Now every .txt file has the artist name.
Ok now for the meat of it! (finally, sorry)
Under the "Tools" tab, click on "Generate Tags with Autotagger on all images"
Inside the popup, you will have a list of all the different autotagger models. DISCLAIMER: I have not tried them all because each listed is a separate 2+Gb download.
The one I use (and gives pretty good results) is "wd-v1-4-moat-tagger-v2". Select it and also select under "Result output mode" click the option to: "Only new with additions to existing" (This will make it so it doesn't OVERWRITE your artist name)
Set "threshold" to 40, leave all else. Click OK
Your GPU will do GPU things, and it's generally a quick process.
ALL DONE!! ... the autotagging. While this model is really cool and picks up on alot of things you'll probably be impressed by, you might need to go in and fix some things. Delete a few lines, shouldn't take too long with those handy shortcuts i told you about... You forgot? It's Ctrl+E to add a line, Ctrl+D to Delete a line. :) I gotchu.
Once you're all good and happy, it's all finished!!! For real this time.
Now you'll need to go back to your dataset with both the resized 2048p pix and .txt files folder, and Ctrl+A to select all. (you have to do it this way INSIDE the folder itself) Select all and right click, compress to .zip
Name it whatever.
STEP 6: CIVITAI AUTOTAGGING.
I've had issues getting this to work, however, In theory it's easier than using the Prompt Manager. Once you have your RESIZED dataset (I don't believe you need to rename them for this) you can .zip your entire dataset without the .txt or drag and drop files loosely into Civitai and it will Autotag. You can set how many tags, a maximum of 30. You can Prepend your artist name. and hopefully if it works well for you, you can do it this way. Your milage will vary. Das it.
PART 7: FINAL STEPS
Go through the steps, naming, select artstyle, etc.
Drag drop your shiny new dataset into the thing after using prompt manager or Civit's autotag.
DISCLAIMER: (I don't know if there was a problem with the servers, but .zip files above 2GB did NOT work for me, your milage may vary. I've had to cut down my dataset cause of this, just a head's up)
Now you'll get to a spot where you can go over your tags, you can add or delete here if you'd like as well.
As for final training.
It depends on how long you want it to cook for/ how much buzz you want to spend. More oven-time= more buzz.
My go-to's are 20 epochs @ 3-4 images trained in VRAM. Around 2-3000 steps. A good medium is 10-12 epochs.
These are the main things to look for, in order Epochs are how many times the model will train. How many images it will train at once and at how many steps to train per epoch.
Other good settings are: shuffle prompts, it will improve learning if you used booru tags and not natural language.
Now it's up to you to decide how many of these settings you want to do and how much you buzz you will pay.
Uhhh that's pretty much it, congrats! If you legitimately read all of this, nice... Fuckin' nerd.
//