Bot & Dolly and Gmunk used robots, performers and projection-mapping to create a mesmerising film that was all captured in-camera.
San Francisco-based Bot & Dolly teamed up with design director Gmunk to create a short film to serve as both an artistic statement and a technical demonstration. 'Box' uses a mix of technologies including large-scale robotics, projection mapping and software engineering, and was captured entirely in-camera.
Tarik Abdel-Gawad on the brief
Before Box, most Bot & Dolly projects employed robotics in film and commercial production. They were used to move cameras, lights, props and actors in order to achieve a certain shot, and the audience didn't see the technical processes we developed and used on-set.
In Box, the process is part of the piece itself. Instead of filming a human subject to add an element in a CG world, Box uses projection mapping to bring CG elements out into the physical world where a live performer can interact with them. Projection mapping onto moving objects unlocks a world of interactive potential, in which a performer's action triggers both real and imagined effects.
Latest Videos From Creative Bloq
The box reacts to touch, gestures and movement in the opening section, transformation
It's natural for me to link the creative and technical processes as I play both roles on projects. For Box, our initial focus was creating a system for projecting onto moving objects. Only after we created that system did I write a treatment based on categories of classical magic. From there we blocked out the animation with Maya and Bot & Dolly's own software, BDMove. Gmunk then joined the team as design director. As an extension of our early process demonstrations - and for mainly practical
Gmunk continues the story...
I joined the Bot & Dolly design team in February and began working on the Box project shortly after. I took the treatment and contributed my design approach to the existing creative. We then focused on the acts that seemed best-suited to provide a coherent editorial structure, narrowing them down to our five favourites - transformation, levitation, intersection, teleportation and escape.
The design approach was informed by black-and-white optical illusion art, set within a film noir aesthetic. Each section needed to evolve into something new, both aesthetically and technically. The design had to be quite minimal as we were projecting artwork onto canvases and re-filming it in a documentary style. Working at a large scale with two layers of optical transfer meant any finer details would get lost.
The team blocked out the animation using Maya and Bot & Dolly’s BDMove software
For the final film, everything was captured live in-camera with no post effects or treatment. Filming took place at Bot & Dolly's studio, with four CG artists creating the projection visuals using Maya, Houdini, Cinema 4D and After Effects. We rendered everything in V-Ray out of Maya to ensure the graphics had a physical and properly illuminated quality of realism.
Get the Creative Bloq Newsletter
Daily design news, reviews, how-tos and more, as picked by the editors.
Each robot held a 4x8 canvas. We needed to synchronise the motion graphics with the robotics animation, which was then tightly synchronised with the lighting. We spent a ton of time on the lighting and re-shot and re-lit things multiple times to get it feeling right, as a big sticking point was how much of the studio to reveal. Lighting designer Phil Reyneri used TouchDesigner for the programmable lighting and also handled all of the projection content.
Graphic extrusions and recesses are used in Intersection – it made editorial sense for this to be the middle section
I suggested we bring Keith Ruggiero in as sound designer for the project, as I'd had the chance to work with him extensively before joining Bot & Dolly. He joined around three months after I did. We developed the background music and made sure it had a nice arc to match the visuals. When we were happy with that foundation, we began adding elements of sound design. We were inspired by Cliff Martinez's work on films like Contagion and Drive, as he produces great ambient works that fit perfectly with the kind of vibe we wanted to create.
Conclusion
We didn't want Box to be over-produced or over-polished. Everything was done in-camera, so you can see mistakes and delays in the projection if you look closely enough - that's all part of allowing it look real.
Objects move from one canvas to another to suggest teleportation
This is definitely the most complex project I've ever worked on. It involved automated cinematography, lighting design, reactive lighting and robotics, not to mention projection on moving surfaces. The end result is a unique piece - everything works together in harmony and that's what makes the piece so captivating. The lighting and graphics are evolving all the time and really pushing a smooth visual and narrative arc. We're now talking to major musicians about using these techniques on stage, and we're looking at doing installations as well.
Words: Tarik Abdel-Gawad and Bradley G Munkowitz
This article originally appeared in Computer Arts issue 223.
