VR: is it the future of design?

The final months of 2016 were a hive of activity in the world of VR. Following launches of the Facebook-owned Oculus Rift and HTC Vive in the spring, PlayStation VR unveiled its eponymous headset in October, Samsung Gear VR got an upgrade, and Google’s Daydream View hit the shelves.

What’s more, in the final days of November, Microsoft announced that its HoloLens – an industry redefining self-contained wearable that intelligently reads the user’s environment and overlays it with 3D holograms – was finally available for developers, with a consumer launch expected some time in 2017.

If 2016 was a watershed moment in terms of the tech, then 2017 is when we’ll see the creative potential fully erupt.

• The 6 best VR headsets for 2017

For creatives working in branding and advertising, the possibilities of VR are sure to get pulses racing. Not only do VR, AR and MR (mixed reality – the term Microsoft uses to describe its HoloLens environment) offer an unparalleled opportunity to provide customers with exciting rare or impossible experiences, or educate them about a product or service in a distraction-free 3D environment, but studies have shown that viewers’ feelings of engagement and empathy are much higher in immersive VR environments than when watching a standard 2D film campaign.

On top of this, there’s huge PR potential because of the relative novelty of the form. But working in such new, unexplored territory requires quite the mental shift. Because of the viewer’s autonomy, the average approach to brand storytelling will wither and die in virtual reality. Plus, it’s expensive.

“Just because you can, doesn’t mean you should,” points out Sol Rogers, founder of virtual reality agency REWIND. A VR experience, he argues, must offer something valuable that justifies the medium. “Always ask why a piece should be in VR, and why it couldn’t just be a TV show or an app. Think about the consumer first, and don’t be seduced by the technology.”

Different realities

VR experiences can be roughly broken down into three categories: active, semi-active and passive. Active VR does what it says on the tin: the viewer can interact with the environment, whether they are solving puzzles, killing baddies, drawing or flying. A 360° video with hotspots where the user can choose to follow a specific character or unlock the next part of the experience is considered semi-active, while passive VR is used to describe 360° video where the viewer can turn their head to watch the action unfold at any compass point, but cannot influence what unfolds in front of them.

Knowing what you want to create, and who you want to create it for will have a massive impact on the platform you choose to design for. At the top of the chain, there’s room-scale real-time engine VR – all singing, all dancing VR platforms that work by hooking up a headset to a game engine, for example the HTC Vive, Oculus Rift and PlayStation VR. It’s here that interactive, processing-hungry experiences work best. The next layer down is mobile VR – that’s platforms such as Samsung Gear VR, Google Daydream and Google Cardboard, which work by slotting a phone into a headset (or in Cardboard’s case, a cheap cardboard viewer) and using the phone to boot the experience, without the need for wires.

Interactive content can work here, but with much less power, and it’s mobile VR where 360° video is best used and produced. The bottom of the ladder – but not an area to be overlooked – is virtual reality outside of the headset, for example Facebook360 and YouTube360. Here, users can explore a 360° environment on their phones, tablets or desktops without owning a headset. 

“You can go upstream very easily, but downstream is very difficult,” states Rogers. “If you make a beautiful thing for HTC Vive, it’s a huge amount of pain to retool it for mobile platforms, and the same again to transform that into 360 video° – you will end up with several different workstreams,” he explains.

Crafting experiences

The biggest USP of VR is the ability to place users in environments – real or fictional – that inspire awe and delight. Just as REWIND gave users access to the International Space Station in Home – A Spacewalk (see left), adam&eveDDB and MPC Creative developed the Buster the Boxer VR, where visitors to John Lewis’ Oxford Street store could don an Oculus Rift headset connected to a Leap Motion detector and Kinect camera and play with the adorable animals – including Buster the Boxer – from John Lewis’ 2016 Christmas ad. Set in a back garden, with lots of brickwork, foliage and fur, the experience was developed in Unreal Engine due to its strength in visual fidelity.

Users can ‘conduct’ the animals to make them jump, accompanied by a subtle melody as well as ambient rustles and growls. Intuitive handtracking was key here to reduce the learning curve for new users. “John Lewis is a family brand,” explains MPC Creative’s interactive creative director, Andre Assalino. “We knew the audience would include kids and grown-ups of all ages, so we didn’t want a complex controller system.”

Another challenge was rendering such rich content at 90 frames per second per eye – something essential in VR to avoid motion sickness. “If there’s any lag, it’ll make you feel really sick,” says Assalino, who also suggests thinking beyond the headset. So other shoppers could understand the experience, MPC Creative developed a 3D camera that captured the player and transposed it on-screen onto their experience in real time. At the end of the game, the player received a receipt featuring a URL of their video.

“VR traditionally isn’t shareable, so that was really important to us,” Assalino adds. On the more fantastical side, Framestore worked with Warner Bros to develop Fantastic Beasts and Where to Find Them VR, an interactive experience with mind-bending VFX for Google Daydream based on JK Rowling’s latest tale. The experience is set within protagonist Newt Scamander’s shed, where players can flick through his books, cast spells using a wand (in reality, the Daydream controller) and even meet magical characters from the film. The project pushed the boundaries of what can be done with mobile VR, and is an excellent example of what’s to come.

The team invented techniques to combine real- time interactive elements with an environment rendered offline, increasing the visual fidelity of the experience and making it all the more believable. Similarly to Buster the Boxer, the strength of this project lay in its strong integration with other marketing strands. “It’s important not to create a VR project in isolation,” states Assalino.

Prompting real feelings

But VR’s power to inspire needn’t just be about spectacle – its ability to generate empathy with real world situations is one of its biggest strengths. “When people put on 360 headsets, they experience film as though they were there,” says Henry Stuart, CEO and co-founder of VR studio Visualise. “If people on camera talk to you, you feel an emotional connection. There’s huge value in that.”

A Walk In Their Shoes, a recent project by AT&T and Mycoskie for ethical shoe brand Toms, involved creating a 360° film for Google Cardboard that follows a Toms customer to Colombia, where the viewer sees how his purchase has helped a child there. The inexpensive viewer was given to Toms customers when they purchased the shoes. Another Google Cardboard project is #prideforeveryone, a global VR Pride parade made for Fundación Sergio Urrego that allows anyone – even in places where homosexuality is illegal – to join in the fun and know that they’re not alone.

This sense of closeness also allows us to explore historical events in an unprecedented way – something sure to interest brands with a strong historical narrative. Excellent recent examples include Remembering Pearl Harbor, a collaboration between TIME’s LifeVR, HTC and Deluxe VR, which follows Lt. Jim Downing through the attack using HTC Vive.

Similarly, Easter Rising: Voice of a Rebel (for the Oculus Rift) is a powerful account of war-torn Dublin in 1916, created by the BBC, Crossover Labs and VRTOV, which uses a distinctive geometric animation style hinged on the idea of reconstruction, a well-crafted narrative and plenty of haunting eye contact. But creating emotive 360° content needn’t be sombre or incredibly high-tech. To see how powerful and infectious positive energy in VR can be, just watch the BBC’s YouTube360 video of Leicester City fans celebrating their Premier League win in a humble pub.

“Build an interesting narrative,” says Visualise’s Stuart. “Cajole the client into that way of thinking. Your experience needs direction and thought, and must give users more value than just looking around.”

In December 2016, Visualise worked with The Financial Times on Hidden Cities: Dublin, a 3D 360° video inspired by the gruesome Emerald Noir crime fiction genre that has emerged in Dublin since the recession. The film was produced for an FT microsite, Facebook and YouTube. In the film, the FT Weekend Magazine’s associate editor Natalie Whittle explores the darker side of the city with award-winning crime writer Tana French, who discusses the places that have inspired her book.

It was a complex shoot – the sweeping cinematic images in low light were shot on a custom-built 3D (stereoscopic) VR rig based on two Sony A7sII cameras with modified lenses, whereas external shots and several time-lapses were filmed using a Google Jump rig. “It’s not a very good advert for Dublin at all,” laughs Stuart. “It’s a brave move for Google too – you’d expect them to want to pin restaurants, but here we have graveyards and Poolbeg power station,” he adds.

Not just a tech demo

Although tech demo-style VR rollercoasters and race tracks have been done to death and are, thankfully, mostly behind us, the idea of the demo shouldn’t be abandoned all together. The immersive nature of VR makes it an excellent education tool, whether you’re showcasing a new product or using VR as a documentary medium.

To launch Jaguar’s I-PACE car, REWIND collaborated with creative agency Imagination to create a real-time multiuser social VR experience for press and VIPs over two continents. Using 66 connected HTC Vive Business Edition headsets, REWIND developed an environment where users in LA and London were able to interact with each other, and then beamed in live video of a presenter talking through the car’s innovations.

Built in Unity, the experience involved 3D film content and live video (including the designer demonstrating the design process using Tiltbrush), and interactive elements where each user could pick apart the engine to see how it worked. There was, of course, a VR test drive – something integral when your prototype car is so precious no one can touch it.

The PR buzz around the launch was phenomenal, but REWIND also launched a light version of the experience to Viveport (the app store for VR) that night, so anyone in the world could experience the fun.

The future of storytelling

The end of 2016 saw some canny acquisitions (The Eye Tribe by Oculus, and Eyefluence by Google) that hint that eye-tracking will be a big addition to VR in 2017. As well as allowing you to shoot lasers out of your eyes, the technology will be a big boon for navigation. It will also be huge in terms of understanding the psychology of a user – the gamer that frequently looks at a weapon, but doesn’t pick it up, for example.

“The reality is that it’s adding another interactive element to VR,” says REWIND’s Sol Rogers, who has worked on projects for FOVE – the first real-time engine VR platform with eye-tracking, which is expected to have a consumer launch later this year. “Plus, for marketing exercises, we can track where you’re looking, what your eyes rested on, and for how long.”

Another benefit is what’s called foveated rendering – the tech that FOVE is named after – where only points where the eyes focus are rendered at high-res, giving a 4x performance boost. This makes high-quality, wire-free VR a real possibility.

But the star of the show has to be mixed reality. “I have said rather boldly that mixed reality is the biggest tech advancement since fire,” says Rogers. “It’s the first thing that augments human intelligence where I can add in as much education, knowledge and training that I want.”

The implications on training and creating in 3D are huge – think complex surgery or 3D modelling – as is the ability to see products like a new kitchen unit or sofa located in situ before you buy. Asobo Studio, a French studio that has developed games for the developer edition of Microsoft HoloLens agrees. “Speaking as a marketing person, I’ve never seen anything like it in terms of involvement,” says Asobo Studio’s communications manager Aurélie Belzanne. “It’s not the same as VR. Imagine you are able to invite your favourite star into your living room, and have them perform a song just for you.”

The games that Asobo Studio has created for HoloLens aim to show developers what can be achieved with the new technology, which features 12 sensors including four environment understanding cameras. Fragments is a life-size first-person crime thriller set in your living room, whereas Young Conker features cute characters that race around the room.

“The magic part is being able to read your environment,” says Asobo Studio founder Martial Bossard. “Knowing a table is a table and how to react to it is the path where no-one has succeeded before.” Bossard’s advice to those that want to explore this exciting new media is to build acclimatisation time into your games, work iteratively and adapt quickly to new paradigms.

“Normally, you show the player what you want them to see, but here the user can look anywhere, not necessarily where you want them to. You need to tease them to look in the right place and be flexible with your storytelling.” In mixed reality, as with VR more generally, the sky really is the limit, and the rules are as yet undefined.

This article was originally published in Computer Arts magazine issue 263. Buy it here!

Related articles:

• The VR web is here
• 5 ways VR is changing web design today
• How VR is revolutionising the creation of 3D art