Meta Unveils AI Model that can Generate Videos from Text Inputs

Facebook parent company Meta has unveiled Make-A-Video, an AI system capable of generating videos from text prompts.

Much of the generative AI focus of late has been on text-to-image generators, like DALL-E, Stable Diffusion and Midjourney. But Meta has instead built a model it claims can “bring imagination to life and create one-of-a-kind videos full of vivid colors, characters, and landscapes.”

Showcasing Make-A-Video, Meta published the model’s work, which included a dog wearing a superhero cape flying through the sky, a spaceship landing on Mars and a horse drinking water. All are short videos similar to GIFs.

Make-A-Video can also create videos from images or take existing videos and create similar new ones.

The AI model “has the potential to open new opportunities for creators and artists,” according to Meta’s research team.

The system was built using publicly available datasets. Meta’s explanation of how the model work reads: “The system uses images with descriptions to learn what the world looks like and how it is often described.

“It also uses unlabelled videos to learn how the world moves. With this data, Make-A-Video lets you bring your imagination to life by generating whimsical, one-of-a-kind videos with just a few words or lines of text.”

Make-A-Video is an expansion to Meta’s prior generative AI work: Make-A-Scene, a multimodal generative AI method capable of generating photorealistic illustrations from text inputs and freeform sketches.

Meta has shared its generative AI research, publishing its research paper on Make-A-Video. The company said it also plans to release a demo experience, though no time frame for this was given.

“We are openly sharing this generative AI research and results with the community for their feedback, and will continue to use our responsible AI framework to refine and evolve our approach to this emerging technology,” a blog post reads.