Simple Stable Diffusion Workflow
Greetings All!
Today I will be sharing one of my workflows with using AI tool Stable Diffusion. I want to be clear this will not be an all encompassing guide on how to use and set up stable diffusion, they are plenty of resources out there that can explain better than I can.
The tools that will be used in this guide are as followed:
- Stable Diffusion (Using the A111 Gui) - Follow this guide to set up (--GUIDE--)
- Photoshop (you can use Gimp or any other photo manipulation software, I am most comfortable with Photoshop, so that is what I will be using.)
Note: I use a merged Stable Diffusion Model combining multiple ckpts, again I will not be going into details as to how this is done, I urge you to use a model that suits a style you want to create, the simple technique I show in this guide can be applied to any model you use.
Step 1: Generate a Composition
Once you have your Stable Diffusion gui setup, you're going to want to generate a composition. I will provide a sample one, with a prompt below:
Prompt: masterpiece, best quality, a close up portrait of a realistic elf girl with red hair and green eyes standing in a forest meadow wearing a long dress tunic, realistic shaded, dungeons and dragons, stunning, intricate, hyper detailed, digital painting, artstation, smooth, hard focus
Negative: deformed, blurry, bad anatomy, disfigured, poorly drawn face, mutation, mutated, extra_limb, ugly, poorly drawn hands, two heads,child, children, kid, gross, mutilated, disgusting, horrible, scary, evil, old, conjoined, morphed, text, error, glitch, lowres, extra digits, watermark, signature, jpeg artifacts, low quality, unfinished, cropped
Seed: 2023501869
Sampling: DDIM 50 Steps
CFG Scale: 11
Step 2: Upscale
Once you have a compositon you're going to want to upscale, reason being I want to disect piece of this composition in photoshop and bring it back into stable diffusion for inpainting.
The upscaling technique I use in Stable Diffusion is 2x and Lanczos, - I use this technique is that alot of upscalers remove too much noise from the original image and leave the composition unpleasant looking. I like to have control of noise removal and sharpening in Photoshop.
Step 3: Photoshop and Inpainting
Now that we have our piece upscaled it's time to bring it into photoshop and chop it up.
This technique will be to take certain parts of the composition and and add more detailed using stable diffusion.
Note: Since this is a simpler guide we will just be cropping and feeding back into stable diffusion, however you take this step MUCH further such overpainting certain details you like or photobashing other elements. They are plenty of tools out there to really push your creativity and composition. I plan on making more tutorials to show such techniques.
First thing we want to do is create a more detailed face, so let's use the marquee tool to make a 512x512 selection of the face area.
The reason we use 512x512, we want to stick to resolutions that Stable Diffusion can inpaint at.
Now with the selection and still on the marquee tool, we want to right click and choose "Layer via Copy" (I want to also note that I am using a Windows machine, if you are on Mac please, google how make such a selection if it's different from Windows.)
Next thing you want to do is right click on the new layer and export it as a PNG. You can name the exported file what ever you like. I named it Face.
Now we want to take the exported file back into Stable Diffusion, and use IMG2IMG In-painting to mask the part we want to generate more detail to.
Now we want to make a new prompt focusing on the area we in-painted.
Prompt: masterpiece, best quality, a detailed face of a woman with green eyes and red hair, realistic shaded, hyper detailed eyes, realistic shaded, dungeons and dragons, stunning, intricate, hyper detailed, digital painting, artstation, smooth, hard focus, illustration
Negative: deformed, blurry, bad anatomy, disfigured, poorly drawn face, mutation, mutated, extra_limb, ugly, poorly drawn hands, two heads,child, children, kid, gross, mutilated, disgusting, horrible, scary, evil, old, conjoined, morphed, text, error, glitch, lowres, extra digits, watermark, signature, jpeg artifacts, low quality, unfinished, cropped
We also want to use the following settings:
Masked Content: Original
Sampling Steps: 50
Sampler: DDIM
512x512 Resolution
11 CFG Scale
Denoising Strength (Note that denoising how much change you want to make the masked content, you want to keep this between .20 to .50, but feel free to experiment with what works for you.)
You can mix different models at this point, as you may have one that makes better faces. Experiment as you like and be free with what you're doing. Also you may save multiple iterations and mask them together in the next step.
This is the face that I will go with, but like I said, you can use multiple iterations using different models and than photobash them together in Photoshop.
Step 4: Masking in Photoshop
Now we want to select the new layer we created earlier, and than drop our newly saved face into the selected box like a jigsaw piece.
Granted you didn't denoise to highly everything should line up perfectly at this point, but let's say you wanted to combine multiple faces or there was slight error that isn't allowing the pieces to allign. This is when we use photoshop's masking.
Take the original layer and put it above the layer with the face you want to use, than create layer mask on the original layer, with the mask layer selected, you want to use a soft brush and black selected for the color in order to blend in the new face you created.
Now you can use this simple technique and and rinse and repeat in different areas of the image, maybe create a new belt? Change her blouse? The path is unlimited and is only restricted by your imagination. With this technique you can make really good pieces, but this is just the tip of the ice berge.
If you learn photobashing and simple painting techniques, you can take this method to the next level. I hope to make more guides in the future in order to add more techniques to your arsenal when creating pieces.
Step 4: Finishing the Image
After you've finshed with your composition, now you can clean it up in Photoshop, I won't go into detail in this step as it can be a seperate guide in itself. What I do essentially is:
- Reduce Noise in the image
- Sharpen Image
- Color Grading
- Manualling fix up imperfections of the image.
Here is the final piece after doing all of the above:
