nateraw/stable-diffusion-videos

Stable Diffusion Videos is a Python library that creates short video clips by smoothly interpolating between AI-generated images. You give it a list of text prompts, and it generates a sequence of images that gradually morph from one description to the next, then stitches them into a video. The example in the README shows a clip that flows from "blueberry spaghetti" to "strawberry spaghetti," producing a slowly shifting visual that transitions between the two. The library builds on Stable Diffusion, an AI image generation model. Instead of producing a single image per prompt, it samples many intermediate points between prompts in the model's internal space, generating a frame for each point. The number of frames between any two prompts is configurable, which controls how slow or fast the transition appears. A music video feature lets you supply an audio file and have the speed of visual changes follow the beat of the music. You define timestamps in the song where you want transitions to occur, and the library calculates how many frames to generate between each transition to match the audio at a given frame rate. There is also a browser-based interface option. Loading it launches a local web page where you can enter prompts and settings without writing Python code, which makes it more accessible for experimentation. Running the library requires a GPU with enough memory to run Stable Diffusion. The code examples use a CUDA-compatible GPU, and there is a note that Apple M1 machines need a slightly different configuration. A Google Colab notebook is provided for people who want to try it without setting up a local environment. The README is brief and consists mainly of code examples. The project was built on top of an earlier script shared by another developer and has since grown into a pip-installable package.