I may have been a little quick to mock two years ago when Meta announced that it was working on an AI video generator with the stunningly unimaginative name of Make-A-Video. Coming on the heels of CEO Mark Zuckerberg's misguided Metaverse obsession, it seemed like another silly way for the company that owns Facebook and Instagram to blow vast amounts of resources researching something that had little to do with its core business and which produced creepy, glitchy video that could barely serve for a meme.
That was in October 2022. Make-A-Video never saw a public release, but Meta continued tinkering away. Now it's just provided a glimpse of its latest AI video generator. And while Movie Gen might have only a slightly better name, its output looks light years ahead of its predecessor. It might even be better than the likes of Runway and Open AI's Sora.
Judging from Meta's Movie Gen research paper, the company's new AI video model can create impressively coherent video that closely adheres to text prompts. But there's more: text prompts can also be used to generate audio in the video, matching music and sound effects with the imagery. The model can also edit existing videos, and still images can be used as references.
Meta says that Movie Gen can create videos of up to 16 seconds in different aspect ratios and audio of up to 45 seconds. It claims that the model compares well against Runway, OpenAI's Sora and Kling, and the few examples provided back that up. Human and animal anatomy, clothing and other objects are generally convincing if still a little fuzzy or overly sharpened and cartoonlike at times. Lighting and shadows look particularly impressive.
Post by @chriscox
View on Threads
And a public release? While OpenAI initially said that Sora would see a public release this year, Meta seems to be in no hurry to release its model. On threads, chief product officer Chris Cox wrote "We aren’t ready to release this as a product anytime soon — it’s still expensive and generation time is too long — but we wanted to share where we are since the results are getting quite impressive."
Sora and Runway have been regarded as the most powerful AI video generators so far, and the developer of the latter has recently entered into a deal to create a bespoke model for the film and TV producer Lionsgate. If it gets a release, Meta's AI video generator is also likely to compete against Adobe Firefly Video, which the company behind Photoshop and Premiere Pro has promised is "coming soon". For the moment, Meta's model looks more powerful, but Firefly Video is likely to have practical advantages like direct use in Premiere Pro.
It's not clear how either model was trained. There have been claims that Runway was trained on videos scrapped from YouTube, which YouTube's CEO has said would be a breach of the platforms terms of use. Meta says that Movie Gen was trained on a “a combination of licensed and publicly available datasets”. It didn't specify which, but Meta has access to vast amounts of video data directly through its own products, Instagram and Facebook.
For more on AI, see our pick of the best AI image generators and the best AI art tutorials.
