Discover real-world solutions for virtual production

Michael T. Morrow is a filmmaker, more versed in real-life documentaries than in computer graphics, digital doubles and virtual production. But when the Covid pandemic effectively closed down his operation, he saw it as a chance to "sharpen the saw, expand the storytelling toolkit".

He began learning a variety of CG apps including ZBrush, Blender and Adobe Substance Painter. With that experience under his belt, when the opportunity came to do a music video for Canadian singer Vicky Rai, Morrow pitched a fairly sophisticated story, involving a hoverbike journey through a sprawling futuristic cityscape and dancing robots.

"The team thought I was insane", he admits. "We have a micro budget, and we don’t have a lot of time. How the hell are you going to do this?"

Michael knew it could be done, however, he just wasn't sure how. "We’ll leap and hope the net appears," he suggested, optimistically.

Creating digital doubles

His first step was to create a digital double of Vicky using Reallusion Character Creator 3, an easy-to-use app for building and rigging digital humans and creatures. The lifelike mesh – complete with photographic texture of Vicky’s face  – was then imported into Reallusion iClone.

iClone is a real-time 3D animation package in which you can hand-craft body and facial animation or generate moves automatically. It also lets you apply motion capture data and setup and render entire scenes.

Creating Vicky’s digital double was pretty intuitive, explains Michael. "I did maybe one tutorial on just the basic interface. Once I got into it, it was like playing in a sandbox. You have to understand that rigging characters, building 3D characters, animating 3D characters… all this is new to me. It’s a daunting task to take a character, rig it from scratch and then animate it. Fortunately in Character Creator 3, every time you make a character it’s already rigged and it’s already weight-painted and it’s ready for animation. To send it to iClone to animate it, you just press a button and it pops over. It’s awesome."

Vicky’s avatar had some mocap dance moves applied and the scene was rendered out. This proof-of-concept turned out better than Michael had expected, and the project was greenlit.

Shooting live-action scenes

Having got the thumbs-up, the team moved to shooting the live action elements, which involved Michael, his director of photography (DOP) Russell Gienapp, makeup artist Cathy-Ann Cuthbert, and the DOP’s daughter, Maia Svenneby, who was in charge of playback. The shot took place in a spacious garage against a green screen, where they captured Vicky on a real motorbike, doubling as the hover version, and doing various dance moves. The footage was edited in Final Cut and green screen removal was achieved in DaVinci Resolve.

Plugins to speed things up

All the CG elements were to be rendered using Unreal Engine, which has been gaining traction as a solution for VFX, for the quality and speed of its output. Sending the iClone character of Vicky to Unreal proved slightly less straightforward, although there are specific tools there too. "There are two plugins that saved my bacon," confesses Michael. “One is the Reallusion Auto Setup for Unreal Engine – which is genius. Once you have the plugins set up in both applications, you hit a button and it ports the character, materials, everything. So you have a one-to-one character in Unreal Engine, the same as you do in iClone. It’s amazing.

"The second plugin I used quite extensively on this project was iClone Unreal Live Link. Once you have the character in Unreal, you connect the two programs together and then any animation you have in iClone will basically be mirrored in Unreal Engine. Then within Unreal you can record those actions. So you get the benefit of all the realtime rendering aspects of Unreal Engine."

By working concurrently in both apps via these plugins, Michael believes it saved him hours if not days of potentially complicated work. "I still had to figure it out", he explains, "but it made the project doable for a single person, with high aspirations, trying to create a project of this scope."

Pre-made animation files

For the big finale, Michael wanted to have a group of robots dancing along with Vicky. Rather than attempt this by hand, he relied on Reallusion ActorCore, a huge repository of affordable motion-captured animations available for multiple platforms. You can purchase individual moves or bundles of themed animations, with everything from casual gestures to zombie attacks. 

“I was able to find everything I needed pretty easily,” he says. “ActorCore is super well-organised and the cool thing is, when you pick your animations, you can choose which software package you want to use. I picked Unreal Engine because many of the robots I’d purchased on the Unreal Engine marketplace, and many of these assets already had the Unreal skeleton associated with them.”

The extensive feature set, intuitive ease-of-use and clever connectivity of apps like Character Creator 3 and iClone have enabled Michael – a documentary filmmaker with little previous experience of CG – to produce a VFX-packed video on a tight schedule and an even tighter budget. 

“A couple of months earlier I had no Idea what I was doing in Unreal Engine and now I’m making robots dance,” he says, somewhat surprised. “The whole process of creating characters in Character Creator 3, through to rudimentary animations in iClone, bringing everything into Unreal Engine. Getting real-time feedback, building a city… these are things I was not able to do a year ago. And with these tools and this ability to create these worlds and create a project of this scale is something that’s really inspiring and really encouraging, and I feel like I’m just scraping the surface. So thank you Reallusion, thank you Epic Games. I look forward to using your tools to create some really cool projects in the future.”

To find out more about Reallusion's tools, visit the Reallusion website.