Fluffusion

NEWS:

R3 is done for now! Uses v-prediction - bring a YAML.

1024x model looking potentially Stable Cascade based.

Anime model is cookin; https://pixeldrain.com/u/dFWmmFq5 bring a YAML. You can use by novelai to simulate the effects of many NAI mixes.

Fluffusion Release:

Fluffusion is a 704 base resolution model with aspect ratio buckets, Clip Skip 2 and effectively unlimited token limits. Now with modern v-prediction.

R3 (704x):

Helpful Negs:

no pupils, empty eyes, derp eyes (If eyes are present. derp eyes is an optional extra.)
black background (Deals with dark images at low CFG rescale.)
blurred background (If you want to avoid the depth of field or photo background effect.)
sketch (If the style isn't sketchy/unfinished, see by dacad for a style that's half sketch half painting.)
comic (Helps reduce panels being generated with more than one character.)
screentone (Reduces noise for smooth, or cel shaded styles.)

Helpful Settings:

CFG 11 seems to be better than CFG 7 - and use at least 0.1 CFG Rescale values.

Models and Utils:

Note: Epoch 13 and up contain the Furception VAE by default - just set VAE to none in Auto to use it, or in Comfy, drag the VAE node from the R3 epochs that contain it. This does not affect images going into the model - only latents converting into normal images. Not images being converted into latents.

R2 (640x):

Notes on E16+ - an auto1111 bug happened where it'd merged R2E16 with R3E1 during testing. I then made a 0.01% merge towards R3 and mostly undone the damage from perturbed noise.

R1:

All known artists in the e6 dataset have been given a prepending of by - so that means that for specific cases like pig (artist) or hoodie (artist) where it'd generate a pig or a hoodie, it will instead actually see them as an artist. So the changes would be more like this by pig (artist) and by hoodie (artist) respectively.

Troubleshooting:

  • My generations look crusty or blurry.
    Your generation resolution is below 640 on the horizontal or vertical axis.
  • The tags feel very heavy handed, and don't seem to work properly?
    Enable Clip Skip 2 in your preferred software settings.
  • My prompts from other models, or merged models don't work properly, what's going wrong?
    NovelAI based prompting like masterpiece, best quality have zero effect on Fluffusion, and Fluff Proto models. For e6 adjacent tags, hi res and absurd res are similar tags. This doesn't apply to R2 or R2HD.

Experiments:

proto r18:

Note: This is a test epoch of the same minified dataset, trained against R1E20 containing Gamma SNR and perturbed noise training at a bucketed resolution of 1280x1280.

proto r11_e1:

Note: This is a test epoch of a minified dataset trained against R1E20 containing Gamma SNR and perturbed noise training on a new seed. Things may be hilariously broken.

r1 test lowLR:

Note: This is an experimental test epoch containing 920k images with their associated tags.

r1 test highLR:

Note: This is another experimental test epoch containing 920k images with their associated tags.

Historical Models:

fluff_proto:

r10:

Edit
Raw PDF PNG
Pub: 22 Mar 2023 19:51 UTC
Edit: 25 Mar 2024 00:42 UTC
Views: 10322
new·what·how·langs·contacts