NOISE INJECTION

WaifuDiffusionV

What is it?

WaifuDiffusionV is a test model released by the WD team which uses a different schedule compared to basic SDXL
https://huggingface.co/waifu-diffusion/wdv-tests

What's with the noise?

When not using the dedicated Node, the model outputs something similar to Pixel Noise.

What is special about this specific noise?

Probably nothing. The process might be replaced with any other model that outputs plain noise but there aren't many (or any, because why would anyone do this?)

What do I do with the noise?

You merge it into your own model or mix at a low weight (0.03-0.05, if Block Merging is available, merge only in the Output blocks. Didn't test which OUT layer is the most important yet but probably one of the early ones OUT1-4)

Why would I want noise in my model?

Noise in model can lead to better backgrounds, more defined lines, better colors and contrast, more added items in an image making it feel more real than '1girl, simple background'.

How do I add the noise to my model?

There are 2 Options:

Download the WaifuDiffusionV test model from https://huggingface.co/waifu-diffusion/wdv-tests (make sure to download the required nodes and play with it, it's looking great up until now for a test model if you don't make pr0n). Use ComfyUI, Load your model and WDV, Use the ModelMergeBlocks node, use your own model as model 1, use WDV as model 2, set ratio to something like 0.97, save checkpoint with the SaveCheckpoint node. (or run some samples on it with a KSampler)
Download the extracted Lycoris for WDV - Base SDXL from https://gofile.io/d/Dsgrfs (Temporary link). Add it to your prompt/workflow at 0.05 strength. Profit!

If you use ComfyUI/Know how to not load a lora on HiRezFix on A1111, you can increase the Lycoris weight to 0.1 as long as it's only on the lowres pass and the denoise of HiRez is ~0.5 or above. This will result in even more noise but it will get solved by the Hirez pass

Other Samples:

Downsides

Hands might be worse, especially if merged into the IN/MID blocks (subject to future testing)

Q&A

Why are you doing this?

Better backgrounds on models that are bad at backgrounds like pony

What is KohakuDayo?

A merge Between Kohaku Epsilon Rev 2 and HoloDayo, also known as LeekAI

The images look burnt

I use a high CFG most of the time ~8 as Euler_smea_dy can handle it on some models. You are free to test it with lower CFG ~6

Why should I do this?

You shouldn't. You should test WDV instead. As a funny tidbit, if you set Lycoris value to 1 and use the WDV Sampler, you can get very different images while keeping parts of your model's knowledge. They might look "bad" but they are definitely interesting.