Creating a web-based GUI for this video generation tool can be done using a web framework like Flask. Here's a simple example of how to create a Flask application with a basic form to input the text prompt and generate the video:
Install Flask in your Conda environment:
conda install flask
Create a new folder named templates inside the sd-video folder. Inside the templates folder, create a new file called index.html and add the following code:
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Video Generator</title>
</head>
<body>
<h1>Video Generator</h1>
<form action="/generate" method="post">
<label for="prompt">Enter your text prompt:</label>
<input type="text" id="prompt" name="prompt" required>
<button type="submit">Generate Video</button>
</form>
</body>
</html>
In the sd-video folder, create a new file named app.py and add the following code:
from flask import Flask, render_template, request, send_from_directory
from sd_video import SDVideo, save_vid
import os
app = Flask(name)
model = SDVideo('model', 'cuda')
@app.route('/')
def index():
return render_template('index.html')
@app.route('/generate', methods=['POST'])
def generate_video():
text_prompt = request.form['prompt']
frames = model(text_prompt)
save_vid(frames, 'output')
Compress the generated frames into a ZIP file
os.system('zip -r output.zip output/*.png')
return send_from_directory('.', 'output.zip', as_attachment=True)
if name == 'main':
app.run(debug=True)
This Flask application has two routes:
/: Renders the index.html form where users can input a text prompt.
/generate: Receives the text prompt from the form, generates the video frames using the provided model, saves them in the output folder, compresses the frames into a ZIP file, and sends the ZIP file as a download.
Run the Flask application:
python app.py
Open a web browser and navigate to http://127.0.0.1:5000. You should see the form to input a text prompt. Enter a prompt and click "Generate Video" to download the generated frames as a ZIP file.
To automatically combine the generated frames into a video, you can use the imageio library with the FFmpeg plugin. This will allow you to create a video file from the generated frames directly.
First, install imageio and the imageio[ffmpeg] plugin in your Conda environment:
conda install imageio
conda install -c conda-forge imageio-ffmpeg
Next, modify the app.py file to include the imageio library and a new function called frames_to_video:
from flask import Flask, render_template, request, send_from_directory
from sd_video import SDVideo, save_vid
import os
import imageio
app = Flask(name)
model = SDVideo('model', 'cuda')
def frames_to_video(frames, output_path):
writer = imageio.get_writer(output_path, fps=24)
for frame in frames:
writer.append_data(frame)
writer.close()
@app.route('/')
def index():
return render_template('index.html')
@app.route('/generate', methods=['POST'])
def generate_video():
text_prompt = request.form['prompt']
frames = model(text_prompt)
save_vid(frames, 'output')
Combine the generated frames into a video
frames_to_video([imageio.imread(f'output/{frame:04d}.png') for frame in range(1, len(frames) + 1)], 'output.mp4')
return send_from_directory('.', 'output.mp4', as_attachment=True)
if name == 'main':
app.run(debug=True)
The frames_to_video function takes a list of frames (NumPy arrays) and an output path, then writes the frames to a video file with the specified path.
In the generate_video function, after saving the frames, the frames_to_video function is called to combine the frames into a video file called output.mp4. The video is then sent as a download.
Now, when you run the Flask application and submit a text prompt, the generated video will be automatically combined into a single video file and offered as a download.