If you are new, don't jump into this yet. Get your feet wet prompting images first and you will have a much easier time understanding what's going on.
Update: Added in the first draft of the Parseq guide and added some info in the Deforum section. I'll have to get the 2D section sorted out.
Greetings anons! This guide is for AI artists who want to emulate animation styles such as anime, cartoons, stop-motion or rotoscope. There are many ways of approaching animation and I want it to be easy for anyone to get into!
This is a living document, I plan on exploring other animation techniques and refining current methods. I hope you anons are willing to glean some info on different processes so everyone can put out cool animations in their own unique styles.
Thank you all for being so supportive and I hope those that are eager to make animations have the time of their lives! I really hope this guide helps spur your imagination. Show us your dreams!
Traditional styles of animation use lower frame-rates. Depending on what kind of animation you want to do, you should have one or multiple selections of fps in mind for different scenes. Below is a chart with the appropriate fps for animation styles:
|Style||fps||fps in post|
|Rotoscope (kind of all over the place, use whatever fits)||8-30fps||30-60fps|
You can consult the performance chart to calculate the amount of time it will take to generate an animation sequence.
shamelessly ripped but very much needed. TY Voldy Guide!
Framerate Calculator here if you need it:
Below is a table of aspect ratios for the height and width options in the Stable Diffusion Webui:
This is a calculator for quick reference as well: https://preyx.github.io/sd-scale-calc/
If you are new to up-scaling, a helpful anon put this together for you:
This also contains useful info for cleaning frames too. Thank you anon!
More in depth information here:
Install ffmpeg: https://ffmpeg.org/download.html
Figured the below would be good to include here since I have other resources listed.
ffmpeg needs to be installed. If its not on your PATH, you may need to manually specify the executable location.
For VP9 webm, ffmpeg must be compiled with libvpx-vp9
For VP8 webm, ffmpeg must be compiled with libvpx
For H.265 mp4, ffmpeg must be compiled with libh265
For H.264 mp4, ffmpeg must be compiled with libh264
~from the Loopback Wave Script Anon's rentry~
As much as I would like to have a fleshed out section for ffmpeg, there are plenty of resources for learning commands. Below are links that provide a GUI for learning them.
Thank you to the anon that put the below link together! Greatly appreciated!
A simple, easy to use input for making webms:
(hasn't been tested on Linux)
A node based GUI used to make ffmpeg filters:
A guide on using prompt interpolation to generate traditional style animations in Automatic1111's Stable Diffusion Webui.
Extremely effective for coherency from examples I've seen but seems really time consuming. The Krita and Photoshop plugins would alleviate a lot of the pain from inpainting in the webui and frame interpolation will cut down on the number of frames you actually need.
So far we have these instructions:
Start with a single vector image.
use any variety of prompts you wish (keep variance per output very low (50-75%).
Using img2img, slowly evolve the type of details you are trying to work on (face, arms, clothing, etc.)
Do this by generating an image based on your current "frame" until you satisfy output to build upon.
You then use that output as your next frame and build from there, gradually adding and removing prompts.
Repeat X times then make a movie.
Example from the same anon:
I will be continuing this section at a later date
Some anons had some luck getting coherent animations but requires a lot of "seed fishing". My experimentation only really gave me one result I kind of liked but that doesn't bring this script off the table. If you have two clips you really want to use with each other and they don't share the same seed (and every other setting is the same ), you can seed travel to the desired seed then fill the frames between clips. Other scripts include seed travel in their repertoire.
A guide for using and understanding the Loopback wave script:
A great primer for techniques in Deforum + Parseq.
A guide to applying Automatic1111's Stable Diffusion Webui to videos with batch img2img.
Really cool and free video editing software and can overlay/animate vector art over the video. You can make your own vector art in the webui using this script:
Some creative anons might be able to make something stylish with this!
Basic overview added will continue to polish and put up more examples in time. I really want to provide camera presets and masks in the future to help out the community.
The most widely used animation script by far. People ask me all the time if I am using it for my animations but I think starting out it's better to learn with simple scripts and apply that knowledge to Deforum. Very good with Img2Img animations. This can also be really useful for pulling off 3D to 2D anime and rotoscoping in general.
Fantastic for plotting animation timings! Sync your animations with music using functions. What I like about this script is the GUI. timeline to track multiple interpolations with wave functions! Love that it's a separate gradio page so you aren't scrolling up and down all the time. Is currently implemented in Deforum (Under Keyframes Tab at the very bottom, also has a link to the web app) but I plan on covering it separately because it uses different functions (see the GitHub documentation). If you have digital music production experience, you will have an easy time picking this up.
Frame interpolation to save time processing extra frames to reach your target fps. Mixed results depending on the video you are trying to interpolate. Illustration styles that use thick outlines and/or fast movements tend to flicker a lot while slow, fluid movements tend to do well.
Parseq is next as I think it is a fantastic way to animate. More Deforum updates coming as well.
Ebsynth Utility for A1111
VToonify: Controllable High-Resolution Portrait Video Style Transfer (HuggingFace demo available).
camera script to record movements in blender and import them into Deforum.
Loopback and Superimpose
An animation focused workflow frontend for Stable Diffusion
Required for the above
Prompt Travel through latent space
High Resolution Depth Maps for Stable Diffusion WebUI
Automate Deforum Keyframe Animations with Waveforms
Collaborative Neural Rendering using Anime Character Sheets