Video is one of the next frontiers for generative AI, and Adobe's just teased some game-changing developments. The creative software giant has given us a glimpse of new AI video editing tools intended to speed up workflows in its widely used software, Premiere Pro.
The new tools will sound familiar to anyone who also uses Adobe's image-editing program Photoshop. They effectively bring some of the key Adobe Firefly-powered tools in Photoshop, like Generative Fill, Generative Expand and Remove, to moving images.
Generative AI in Premiere Pro powered by Adobe Firefly | Adobe Video - YouTube
The new AI tools in in Premiere are intended to allow video editors to quickly and easily extend a shot or add or removing objects in a scene. They will be powered by a new video model that will join Adobe's Firefly Ai suite, which already includes Image, Vector, Design and Text Effects. The headline new tools coming later this year are:
Generative Extend: uses generative AI to add frames to make clips longer. Adobe says the breakthrough technology solves a common problem professional editors run into every day, allowing them to create extra media for fine-tuning edits, to hold on a shot for an extra beat or to better cover a transition.
Object Addition & Removal: allows users to select and track objects, then replace them. We'll be able to remove unwanted items, change an actor’s wardrobe or quickly add set dressings like a painting on a wall or flowers on a desk.
Text to Video: allows the generation of entirely new footage directly within Premiere Pro by typing a text prompt or uploading reference images. These clips can be used to ideate and create storyboards, or to create B-roll for augmenting live action footage.
Premiere Pro's Add object is a Photoshop Generative Fill for video (Image credit: Adobe)
However, Adobe also previewed something that could be coming further in the future. And it's a different, potentially more controversial approach to what Adobe has been doing with still images. The company is exploring the option of bringing third-party generative AI from OpenAI, Pika Labs and Runway into Premiere Pro.
Get the Creative Bloq Newsletter
Daily design news, reviews, how-tos and more, as picked by the editors.
Adobe has so far sought to distance itself from other developers of generative AI tools by presenting Firefly as a more 'ethical' and commercially safe AI model because it was trained on public domain assets and licensed images from Adobe Stock rather than on material scraped from the web without consent.
Last week, there was some controversy as it was confirmed that Firefly's training data contained images on Adobe Stock that were generated by the AI model Midjourney. Now with video, Adobe seems to be taking a route that others like Canva have been exploring, allowing other AI models to be used within its software.
Adobe is exploring the integration of third-party AI video generators like OpenAI's Sora in Premiere Pro (Image credit: Adobe)
Adobe says in its press release: "Creative Cloud has always had a rich partner and plugin ecosystem, and this evolution expands Premiere Pro as the most flexible, extensible professional video tool that fits any workflow. Adobe customers want choice and endless possibilities as they create and edit the next generation of entertainment and media."
It should be clarified that Adobe describes these as "early explorations", and there's no timeline for when such integrations will be publicly available. However, the video preview shows how video editors could, in the future, use AI video generation models like OpenAI Sora and Runway in Premiere Pro to generate B-roll to edit into their project. It also shows how Pika Labs could be used with the Generative Extend tool to add a few seconds to the end of a shot.
Adobe said: "While much of the early conversation about generative AI has focused on a competition among companies to produce the “best” AI model, Adobe sees a future that’s much more diverse."
It stressed that it developed its own AI models with a commitment to responsible innovation and that it "plans to apply what it's learned to ensure that the integration of third-party models within its applications is consistent with the company’s safety standards". As one of the founders of the Content Authenticity Initiative, it also pledges to attach Content Credentials to assets produced within its applications.
Adobe had also announced upcoming general availability of AI-powered audio workflows in Premiere Pro, including new fade handles, clip badges, dynamic waveforms, AI-based category tagging and more, from May.
Interactive fade handles will allow editors to create custom audio transitions faster by dragging clip handles to create audio fades, while the Essential Sound badge with audio category tagging will automatically tags audio clips as dialogue, music, sound effects or ambience and adds a new icon so editors get one-click, instant access to the right controls.
Effect badges are new visual indicators that make it easier to see which clips have effects, quickly add new ones, and automatically open effect parameters from the sequence. Finally, redesigned waveforms intelligently resize as the track height changes on clips. An AI-powered Enhance Speech tool, intended to remove unwanted noise and improves poorly recorded dialogue, has been generally available since February.
See the best prices on Adobe Premiere Pro in your region below, or see our regular guide to Creative Cloud discounts.
Thank you for reading 5 articles this month* Join now for unlimited access
Joe is a regular freelance journalist and editor at Creative Bloq. He writes news, features and buying guides and keeps track of the best equipment and software for creatives, from video editing programs to monitors and accessories. A veteran news writer and photographer, he now works as a project manager at the London and Buenos Aires-based design, production and branding agency Hermana Creatives. There he manages a team of designers, photographers and video editors who specialise in producing visual content and design assets for the hospitality sector. He also dances Argentine tango.
