Adobe Launches New Generative AI Video Tools

Adobe has announced new generative AI video capabilities and other creative tools powered by its Firefly generative AI models during its annual Adobe MAX 2024 conference held from Oct. 14-16 in Miami Beach.

Video Creation Tools

The Firefly Video Model video tools, available in limited public beta, are Generative Extend for editing and Text to Video & Image to Video for video creation. 

Generative Extend enables users to extend clips to cover gaps in footage, smooth out transitions or hold on shots longer to improve video edit timing. Adobe has packaged it into its video editing app, Premiere Pro.

Text to Video & Image to Video offers new text-to-video and image-to-video capabilities, generating video from text prompts, with enhanced angle, motion and zoom camera controls. It also creates reference images for B-Roll generation. The image-to-video capabilities animate still shots or illustrations to create live-action clips. 

Commercially Safe

Adobe claimed at the event that Firefly is the industry’s first “commercially safe” generative video model, meaning that it has been trained exclusively using licensed and public domain content, including Adobe stock images.

“The usage of Firefly within our creative applications has seen massive adoption and it’s been inspiring to see how the creative community has used it to push the boundaries of what’s possible,” said Ely Greenfield, chief technology officer, digital media at Adobe.

Related:Adobe Updates Terms of Service to Pledge No AI Training on User Data

“We’re thrilled to bring creative professionals even more tools for ideation and creation, all designed to be commercially safe.”

AI companies are increasingly under scrutiny for scraping content from the internet without crediting or compensating creators. Getty Images, for example, sued Stable Diffusion parent Stability AI for training its AI model on its images without permission.

Design Tools

Adobe also unveiled enhanced generative tools for still image creation, including the new Firefly Image 3 Mode, which the company said generates images up to 4x faster than previous models.

Generative Workspace for Photoshop supports designers to collaboratively work on new ideas via a new, more intuitive interface. It is powered by Adobe Firefly and is available in beta.

The Firefly Vector Mode enhancements for Illustrator, also in beta, build on the Generative Shape Fill, Generative Recolor and Text to Pattern tools released earlier this year. Users can now control the density of elements in a single pattern to change how tightly the elements are packed together.

Related:Getty Sues Stable Diffusion Parent in the U.S.

Adobe also previewed Project Concept, which brings AI to collaborative “mood-boarding” to the early stages of design projects. 

Dubbing and Lip Sync

Adobe showcased Firefly Services at the event, a creative and generative API tool set. The Dubbing and Lip Sync tool uses generative AI to translate dialog spoken in videos into different languages while maintaining the sound of the original voice and synching lip movements. 

Bulk Create is designed to support editing large volumes of images more efficiently, streamlining tasks such as resizing or background removal.