Kling, Sora and Runway have been the main names to watch for AI video generation, but Adobe has just leapt into the pack with the beta release of its Firefly AI video model. And, based on the demo I was shown, my first impression is that anything it may lack in realism it makes up for in the amount of control it provides, making it potentially the most useful AI video generator for professionals.
Firefly AI video can generate video from text prompts and from still image references. And like with its Firefly image model, Adobe claims it's the industry’s first "commercially safe AI video generation model to market".
Adobe's been teasing Firefly video for a while, but until today, its only publicly available generative AI video tool was Generative Extend in Premiere Pro. That allows users to add extra seconds to existing footage. Now fully blown AI video generation is available, not within Premiere Pro, but in a new-look Adobe Firefly web app.
According to Adobe, the Firefly Video Model generates IP-friendly video content that can be used in commercial productions without fear of copyright infringement (though it hasn't provided much info on training data). For now, it can generate up to five seconds of footage at a time in 1080p resolution at a default 24 fps in vertical or horizontal format. Each generation takes about 90 seconds. Adobe says a lower resolution model for high-speed ideation and a 4K model for professional production are coming soon.
It looks like the biggest advantage of Firefly's video model is the amount of control it provides – something that's been difficult in other models. Adobe's recognised that being able to direct the output is critical to producing usable footage. You can choose shot size (close up, medium, long etc), camera angle (aerial, eye-level, high, low, top-down) and camera motion (zoom in, tilt, handheld, etc) to provide more control over framing and the dynamism of a shot.
The image-to-video functionality allows two images to be uploaded to define the first and last frames of a shot, with the AI generating the movement in between. And like with many image generators, seed number can be used for more consistency in iterations, for example to keep a similar piece of footage with a different camera angle.
Since Firefly already provides still image generation, Adobe envisages a workflow in which users will generate images in Firefly, edit them in Photoshop, turn them into video in the Firefly video model and then take them into Premiere Pro to edit and integrate them into video projects. This complete workflow, benefiting from the powerful colour and masking tools already in Premiere Pro, could give Adobe the edge over rival tools (see our Adobe software list).
When the Firefly image model was released, its output often looked less realistic than results from the likes of Midjourney and Stable Diffusion, which were trained with unlicensed material. I've not seen enough from Firefly video yet to make a call, but initial results don't look so far from other models. A lot of the example footage from Adobe is very clearly AI – those whales (below) aren't going to fool anybody. But the more precise controls make for some strong use cases for Firefly.
Assets can be generated with green screen backgrounds to be ultra-keyed in Premiere Pro or After Effects or on black screen to use with a blending mode to add atmospheric effects like snow, rain and smoke cheaply and easily. The tool could also be really useful for generating temporary fillers to convey creative direction while editors wait for the final footage to arrive.
New Firefly web app
To coincide with the addition of the video model in beta, the Firefly web application has been revamped to incorporate access to the new tool.
Adobe has also added another new AI image tool: Scene to Image in beta. This allows users to design a basic 3D structure in browser and then add a text prompt to turn it into an image.
There's also a new Translate Video tool, which Adobe reckons this will save time on dubbing and audio mixing, Just don't think I would rely on it without a native speaker on hand to verify the results. We're told an AI speech enhancer and text-to-avatar tool will be added soon.
New Adobe Firefly AI plans
New models mean new costs. The Firefly video model is free to try, but with limited generations. Adobe's announced the launch of two Firefly monthly subscription plans: Firefly Standard and Firefly Pro. It seems that existing Creative Cloud customers will still need a plan.
Firefly Standard starts at $9.99 for 2,000 video/audio credits per month, good for up to 20 five-second 1080p video generations.Firefly Pro starts at $29.99 per month for 7,000 video/audio credits – up to 70 five-second 1080p video generations. Coming later, there will also be a Firefly Premium plan for those who want to generate video on a more regular basis.
The new tools are available via the Adobe Firefly website. See Creative Cloud pricing below.
For more AI video news, see the viral videos being generated with Tencent's Hunyuan and the Netflix-linked AI video generator that allow cut-and-drag animation. And for more updates from Adobe, check out the new Project Concept mood boarding tool.
