Stable Diffusion Adds Video Generation

Generate high-resolution videos from text and images – but Stable Video Diffusion is currently restricted to researchers

Ben Wodecki, Jr. Editor

November 23, 2023

2 Min Read
Stable Video Diffusion generations. Stabilty AI, who made Stable Diffusion, developed a new AI video generation model using text and image inputs.
Reserved for researchers, Stable Video Diffusion 'not intended' for commercial use yetStability

At a Glance

  • Stability unveils Stable Video Diffusion, a new video generation AI model set to rival models from Runway and Pika Labs.

Stable Diffusion maker Stability AI has unveiled its first video generation model based on the popular text-to-image system.

The company revealed Stable Video Diffusion, a generative AI video model that can create videos from text prompts.

Simply type 'a rocket taking off in the desert' or 'waves crashing against the shore’ and Stable Video Diffusion will create the desired output.

The model can also generate videos from still images. According to the Stable Video Diffusion paper, the team behind it designed the model to ensure high-resolution image-to-video modeling.

It’s designed for tasks like multi-view synthesis from a single image – so animators could use it to generate different camera angles of an object or help build 3D environments for VR and AR experiences.

How to access Stable Video Diffusion

The model is currently restricted purely for research. A Stability blog post said the model “is not intended for real-world or commercial applications at this stage.”

Instead, the team behind it is seeking feedback on safety and quality to refine the model for an eventual release.

Researchers can access the code for Stable Video Diffusion via GitHub. The weights required to run the model locally can be found on Hugging Face.

Stable Video Diffusion is available via two image-to-video models, capable of generating 14 and 25 frames at customizable frame rates between three and 30 frames per second.

Related:AI Image-Generation Models and Tools: The Ultimate List

However, in early signs of its abilities, Stability performed user preference studies and found that Stable Video Diffusion was preferred to rival models from Pika Labs and Runway for video generation.

Graph showing results of a user preference survey on AI video generation tools. The Stable Video Diffusion model was the top preferred model compared to Runway and Pika Labs

You can sign up for the waitlist to access an upcoming web experience featuring a text-to-video interface that showcases practical applications of Stable Video Diffusion across education, marketing and entertainment. On the contact form, select 'Stable Video – Waitlist' in the drop-down menu.

Read more about:

ChatGPT / Generative AI

About the Author(s)

Ben Wodecki

Jr. Editor

Ben Wodecki is the Jr. Editor of AI Business, covering a wide range of AI content. Ben joined the team in March 2021 as assistant editor and was promoted to Jr. Editor. He has written for The New Statesman, Intellectual Property Magazine, and The Telegraph India, among others. He holds an MSc in Digital Journalism from Middlesex University.

Keep up with the ever-evolving AI landscape
Unlock exclusive AI content by subscribing to our newsletter!!

You May Also Like