FizzleDorf's Animation Guide

If you are new, don't jump into this yet. Get your feet wet prompting images first and you will have a much easier time understanding what's going on.

Update: Added in the first draft of the Parseq guide and added some info in the Deforum section. I'll have to get the 2D section sorted out.

Index

Introduction

Greetings anons! This guide is for AI artists who want to emulate animation styles such as anime, cartoons, stop-motion or rotoscope. There are many ways of approaching animation and I want it to be easy for anyone to get into!

This is a living document, I plan on exploring other animation techniques and refining current methods. I hope you anons are willing to glean some info on different processes so everyone can put out cool animations in their own unique styles.

Thank you all for being so supportive and I hope those that are eager to make animations have the time of their lives! I really hope this guide helps spur your imagination. Show us your dreams!

Frame-rate

Traditional styles of animation use lower frame-rates. Depending on what kind of animation you want to do, you should have one or multiple selections of fps in mind for different scenes. Below is a chart with the appropriate fps for animation styles:

Style fps fps in post
Anime (Budget) 8fps 16fps
Anime 12fps 24fps
Cartoons 12-15fps 24-30fps
Stop-Motion 4-25fps 8-30fps
Rotoscope (kind of all over the place, use whatever fits) 8-30fps 30-60fps

You can consult the performance chart to calculate the amount of time it will take to generate an animation sequence.

PerfChart shamelessly ripped but very much needed. TY Voldy Guide!

Framerate Calculator here if you need it:
https://www.zapstudio.net/framecalc/

Width and Height

Below is a table of aspect ratios for the height and width options in the Stable Diffusion Webui:

Aspect ratio

This is a calculator for quick reference as well: https://preyx.github.io/sd-scale-calc/

If you are new to up-scaling, a helpful anon put this together for you:

https://pastebin.com/8WVyDxt9

This also contains useful info for cleaning frames too. Thank you anon!

More in depth information here:

https://rentry.org/sdupscale

FFMPEG

Figured the below would be good to include here since I have other resources listed.

ffmpeg needs to be installed. If its not on your PATH, you may need to manually specify the executable location.
For VP9 webm, ffmpeg must be compiled with libvpx-vp9
For VP8 webm, ffmpeg must be compiled with libvpx
For H.265 mp4, ffmpeg must be compiled with libh265
For H.264 mp4, ffmpeg must be compiled with libh264

~from the Loopback Wave Script Anon's rentry~

As much as I would like to have a fleshed out section for ffmpeg, there are plenty of resources for learning commands. Below are links that provide a GUI for learning them.

Thank you to the anon that put the below link together! Greatly appreciated!

A simple, easy to use input for making webms:
https://ffmpeg.party/webm-from-image-sequence/
(hasn't been tested on Linux)

A node based GUI used to make ffmpeg filters:
https://ffmpeg.guide/

Prompt-Interpolation

A guide on using prompt interpolation to generate traditional style animations in Automatic1111's Stable Diffusion Webui.

Guide: https://rentry.org/AnimAnon-PromptInterp

example:
Imgur

Frame-by-frame Animation (haven't attempted yet)

Extremely effective for coherency from examples I've seen but seems really time consuming. The Krita and Photoshop plugins would alleviate a lot of the pain from inpainting in the webui and frame interpolation will cut down on the number of frames you actually need.

So far we have these instructions:

Start with a single vector image.
use any variety of prompts you wish (keep variance per output very low (50-75%).
Using img2img, slowly evolve the type of details you are trying to work on (face, arms, clothing, etc.)
Do this by generating an image based on your current "frame" until you satisfy output to build upon.
You then use that output as your next frame and build from there, gradually adding and removing prompts.
Repeat X times then make a movie.

~Anon

Example from the same anon:
Frame-by-frame example

I will be continuing this section at a later date


Seed Travel

Some anons had some luck getting coherent animations but requires a lot of "seed fishing". My experimentation only really gave me one result I kind of liked but that doesn't bring this script off the table. If you have two clips you really want to use with each other and they don't share the same seed (and every other setting is the same ), you can seed travel to the desired seed then fill the frames between clips. Other scripts include seed travel in their repertoire.

Link: https://github.com/yownas/seed_travel


Loopback Wave

A guide for using and understanding the Loopback wave script:

Link: https://rentry.org/AnimAnon-LoopbackWave

Imgur

A great primer for techniques in Deforum + Parseq.


Rotoscoping

A guide to applying Automatic1111's Stable Diffusion Webui to videos with batch img2img.

https://rentry.org/AnimAnon-Rotoscope


ChaiNNer

Really cool and free video editing software and can overlay/animate vector art over the video. You can make your own vector art in the webui using this script:
https://github.com/GeorgLegato/Txt2Vectorgraphics
Some creative anons might be able to make something stylish with this!

link: https://github.com/chaiNNer-org/chaiNNer


Deforum

Basic overview added will continue to polish and put up more examples in time. I really want to provide camera presets and masks in the future to help out the community.

The most widely used animation script by far. People ask me all the time if I am using it for my animations but I think starting out it's better to learn with simple scripts and apply that knowledge to Deforum. Very good with Img2Img animations. This can also be really useful for pulling off 3D to 2D anime and rotoscoping in general.

Guide: https://rentry.org/AnimAnon-Deforum

Example:

Deforum Test


Parseq

Fantastic for plotting animation timings! Sync your animations with music using functions. What I like about this script is the GUI. timeline to track multiple interpolations with wave functions! Love that it's a separate gradio page so you aren't scrolling up and down all the time. Is currently implemented in Deforum (Under Keyframes Tab at the very bottom, also has a link to the web app) but I plan on covering it separately because it uses different functions (see the GitHub documentation). If you have digital music production experience, you will have an easy time picking this up.

https://rentry.org/AnimAnon-Parseq

Example:
Imgur


Flowframes

Frame interpolation to save time processing extra frames to reach your target fps. Mixed results depending on the video you are trying to interpolate. Illustration styles that use thick outlines and/or fast movements tend to flicker a lot while slow, fluid movements tend to do well.

Link: https://nmkd.itch.io/flowframes


Moving Forward

Parseq is next as I think it is a fantastic way to animate. More Deforum updates coming as well.

Ebsynth Utility for A1111

https://github.com/s9roll7/ebsynth_utility

VToonify: Controllable High-Resolution Portrait Video Style Transfer (HuggingFace demo available).

https://github.com/williamyang1991/VToonify

Latent Blending

https://github.com/lunarring/latentblending

camera script to record movements in blender and import them into Deforum.

https://github.com/micwalk/blender-export-diffusion

Loopback and Superimpose

https://github.com/DiceOwl/StableDiffusionStuff

An animation focused workflow frontend for Stable Diffusion

https://github.com/amotile/stable-diffusion-studio

Required for the above

https://github.com/amotile/stable-diffusion-backend/tree/master/src/process/implementations/automatic1111_scripts

Prompt Travel through latent space

https://github.com/Kahsolt/stable-diffusion-webui-prompt-travel

High Resolution Depth Maps for Stable Diffusion WebUI

https://github.com/thygate/stable-diffusion-webui-depthmap-script

Automate Deforum Keyframe Animations with Waveforms

https://www.framesync.xyz/

Collaborative Neural Rendering using Anime Character Sheets

https://github.com/transpchan/Live3D-v2

Contact:

Discord: Fizzledorf#9223
Twitter: https://twitter.com/FizzleDorf
Youtube: https://www.youtube.com/channel/UCdkuTpXmJvHzbxnuszEiOKg
GitHub: https://github.com/FizzleDorf/AnimationGuide

Edit
Pub: 02 Nov 2022 18:18 UTC
Edit: 15 Jan 2023 03:33 UTC
Views: 20361