Shitty SDXL loras
Finetune-extracted styles (Illustrious v0.1)
Random notes
- Followed anons rentry to start with, experiments resulted in this boomer .ps1 script which uses about 22gb VRAM.
- Consistently got underbaked results with (1x dataset + 48 epochs) -> switched to (5x dataset + 16 epochs).
- "Trigger" tokens seem very beneficial even if not used in prompt, didn't really do a thorough test of this
- Haven't messed with B-LoRA slicer for these extracts (yet)
- Timestep fuckery using "shift" is a huge positive for styles, using sd-scripts dev branch with these merged in: timestep shit from sd3 branch + soft snr gamma
- Biggest change while testing was to move away from --full_bf16, likely a skill issue but I needed more epochs to reach the same results compared to having it off despite tinkering with learning rates. This was with adafactor but I assume lion or some optimi implementation with kahan_sum would fare better? Was too lazy to test more once I got decent results from this setup after weeks of messing with lora bakes.
- EDIT: full bf16 seems decent enough if you yoink this into your sd-scripts library folder to have automatic stochastic rounding for adafactor
Mosha style lora, 67 image dataset w/ 5x repeats
1 kept token: artmosha
Training .ps1 script
Mosha v1 e12 64d 32c fixed 459mb- Mosha v1 e12 64d 32c fixed 346mb Smaller version without TE thanks to anons bodged extract script
B-LoRA—inspired styles (Pony Diffusion V6 XL)
Styles baked with a custom preset using Lycoris and then stripped down with blora_slicer.py for compatibility. First use takes pretty long at least on my forge setup but it'll be normal afterwards. The goal was to keep a good balance of style and compatibility with character/concept loras and base pony characters etc.
Deadnoodle style lora
Random anons dataset, didn't test this very much
Trained with score_9, source_anime
Made in Abyss manga style lora
Trained with score_9, source_anime
B-LoRA—inspired styles workflow (Pony Diffusion V6 XL)
Just the basic workflow, I have no clue what dataset sizes, learning rates, algos etc. are actually the best.
This set of sliced traits is just what I ended up with, it may not be optimal for everything. Do some experiments!
Training (sd-scripts):
- Copy this blora_conv_ffnet_outallbut0_inall_mid.toml file to use as a preset during training, you'll need lycoris.
- Example .ps1 file using LoKr algo,Uses roughly 15-18gb VRAM, could be optimized more. You can increase the factor to 3 or 4 if you're desperate, I saw improvements going from 4 to 2 in early testing but I haven't tried again with final settings.
- The .ps1 above uses immiscible noise, so you'll have to merge this pull request https://github.com/kohya-ss/sd-scripts/pull/1395:
Slicing:
- Clone https://github.com/ThereforeGames/blora_for_kohya to your sd-scripts folder, the repo has more detailed install/usage instructions.
1.1 Replace the slicer with this modified one blora_slicer.py if you want the lora to retain it's metadata. - Add this to your blora_traits.json
-
Run the slicer on your chosen epochs using the above --traits and you're done!
python blora_slicer.py --loras ./input/{file} --traits out012345_in12356_te1_te2 --debug --output_path=./output/{no_safetensors}_out0to5_in12356_te1te2.safetensors
Personally I add "input" and "output" folders to my blora folder and use this handy python script folders_blora_slicer.py to run the script on all the loras in the input folder. It's very useful when you're testing multiple presets and multiple epochs.
Styles (Pony Diffusion V6 XL)
Trained with score_9, source_anime
Mosha with new oven settings for baking, sketch is a good tag to use. V9 with shittier tagging was somehow superior at 1girl, standing but had some other issues with prompting. Hands often have issues in both.
Trained with score_9, source_anime, eu03
Quick bake with the 5th eu03 dataset from anon, mostly for science. e16 is probably the one you want.
Trained with score_9, source_anime, tsukushi akihito
9th epoch is probably the best for normal use. Last epoch colors/shading go a bit too hard but it's great for spooky stuff. 1.0 weight works fine but going down to 0.8 brings back more details to the backgrounds.
Trained with score_9, source_anime, moshimoshibe
Shitty loras
Styles (LoKr + Pivotal)
Requires webui 1.7.0 / dev branch
Feels more forgiving to bake while retaining prompt responsiveness & less spaghetti fingers with pivotal? Including the token in prompt seems beneficial but might be cope.
- Raita (token: raita)
- Has some issues with random impact effects at 1 weight, cba fixing the tags and rebaking.
- example
Styles (LoKr)
Styles
Styles (hires)
Styles baked at 1200+ resolution, using these will generate mustard dolphin gas
Concepts
Instant loss 2koma mating press, 2e4 adam and 2e6 ada. Which one is better? It's all gacha bullshit that depends on model/prompt/loras.
2e4 feels gentler on style and 2e6 feels a bit more consistent. I uploaded both so you can share the pain of indecision. More notes in metadata, tldr: controlnet good.
- Instant loss 2koma mating press 2e4
- Instant loss 2koma mating press 2e6
- Instant loss 2koma mating press embed