Stable Diffusion Adds Video Generation

Stable Diffusion maker Stability AI has unveiled its first video generation model based on the popular text-to-image system.

The company revealed Stable Video Diffusion, a generative AI video model that can create videos from text prompts.

Simply type 'a rocket taking off in the desert' or 'waves crashing against the shore’ and Stable Video Diffusion will create the desired output.

The model can also generate videos from still images. According to the Stable Video Diffusion paper, the team behind it designed the model to ensure high-resolution image-to-video modeling.

It’s designed for tasks like multi-view synthesis from a single image – so animators could use it to generate different camera angles of an object or help build 3D environments for VR and AR experiences.

How to access Stable Video Diffusion

The model is currently restricted purely for research. A Stability blog post said the model “is not intended for real-world or commercial applications at this stage.”

Instead, the team behind it is seeking feedback on safety and quality to refine the model for an eventual release.

Researchers can access the code for Stable Video Diffusion via GitHub. The weights required to run the model locally can be found on Hugging Face.

Stable Video Diffusion is available via two image-to-video models, capable of generating 14 and 25 frames at customizable frame rates between three and 30 frames per second.

Related:AI Image-Generation Models and Tools: The Ultimate List

However, in early signs of its abilities, Stability performed user preference studies and found that Stable Video Diffusion was preferred to rival models from Pika Labs and Runway for video generation.

Credit: Stability

You can sign up for the waitlist to access an upcoming web experience featuring a text-to-video interface that showcases practical applications of Stable Video Diffusion across education, marketing and entertainment. On the contact form, select 'Stable Video – Waitlist' in the drop-down menu.