3D scanning is evolving, here's how Sony Digital Imaging helps its customers in film and gaming to deliver ultra-realism.
(Image credit: Sony)
3D scanning is evolving and Sony is leading the way. The field of photogrammetry – capturing high-definition 3D renders of objects, buildings and people, using images stitched together – has taken great leaps forward in recent years in line with camera technologies and computing capabilities.
There’s no doubt that it now constitutes its own specialist discipline. From data capture for video gaming assets to virtual production environments for film, TV and XR, the potential applications of photogrammetry are endless. And with the technical advances, we’re seeing the rise of digital humans: accurate or modified renders of real people that can be used in all manner of settings.
In recent years, Sony Digital Imaging has been making waves in the photogrammetry arena by supplying cutting-edge camera technology that allows their customers to capture renders with a level of realism previously unseen in 3D scanning. The manufacturer prides itself on building deep technical relationships throughout the implementation process that deliver startling results.
3D scanning: Peris Digital
Spanish firm Peris Digital is one of Sony's biggest clients, the studio's work has been seen in Netflix hit Bridgerton (Image credit: Sony / Peris Digital)
Sony’s roster of photogrammetry clients includes the Spanish firm Peris Digital. It’s a child company of Peris Costumes, who have been costume designers and curators since 1856.
Peris create digital scans of any garment, material or texture from any period, borrowing from their parent company’s archive of over seven million examples, for use in TV and film. Their clothing models have featured in film and TV productions for producers such as Netflix and Apple to name just two. Recent screen credits include the movie Dune, the TV series Bridgerton, La Casa de Papel (Money Heist), and Emily in Paris.
Peris have chosen to pursue photogrammetry over CGI because it guarantees the highest level of quality and agility needed to deliver visual FX. The optimal quality achieved through photogrammetry allows assets to be used on forefront characters, and not just with background scenes and extras.
Get top Black Friday deals sent straight to your inbox: Sign up now!
We curate the best offers on creative kit and give our expert recommendations to save you time this Black Friday. Upgrade your setup for less with Creative Bloq.
Photogrammetry instead CGI produces more accurate and detailed models (Image credit: Sony / Peris Digital)
Peris uses Nvidia software at all stages of their product creation, as well as best-in-class Nvidia computers to handle their VFX. Peris featured as a keynote exhibitor in Nvidia's GTC event in November last year.
When it comes to VFX, Peris is an increasingly major player because of their high-end technology and their extensive experience in movie production. The company is also looking to offer the services of their photogrammetry rig to the gaming industry, but for now, VFX for TV and film is their central offering.
Creating a character for screen from scratch with costumes is a very time-consuming process, but Peris Digital can achieve remarkable results in just two-three hours by scanning existing materials and garments. To achieve all this, they use a main rig composed of 144 Sony Alpha 7C and 7R IV cameras, along with Sony’s G Master lenses, to generate 3D models ready to use in post-production work. They also have a smaller mobile rig which they can transport around the world to be used on-set.
Materials and costumes can be scanned in under three hours (Image credit: Sony / Peris Digital)
Garments can have an animated usage – both traditional animation or motion capture – or can be used with crowd software. Garments, accessories and props are available immediately and remotely, and Peris’ clients can access their clothing catalogue with no issues around stock shortage.
One of the challenges that Peris Digital faces is retopology – which is rebuilding a scan after the 2D images have been captured. Peris' artists must inform the software what kind of material it’s looking at, whether it’s cloth, metal and so on. This is a manual job at the moment, but the Peris team is working on automatic texture recognition that in future will do this job unaided.
Sony have played a critical role in helping Peris achieve its goals, beyond just supplying cameras
Resolution is the real key to what Peris delivers. The company is able to replicate the unique texture of garments to make the end results truly lifelike – an important advantage that photogrammetry has over conventional CGI. The Alpha 7R IV is Sony’s highest resolution model achieving 61MP, but the Alpha 7C is also the smallest and highest resolution in its category – and size really does matter on a rig with 144 cameras.
Sony has played a critical role in helping Peris achieve its goals, beyond just supplying cameras. In particular, Sony has helped extensively with integration of the cameras into the rig, and the vital component of camera synchronisation. Sony’s Camera Remote SDK is pivotal in delivering this. Sony also provide the G Master lenses (including the 135mm f1.8) that achieve the crystal-clear images needed for Peris’ work.
3D scanning: Stasis Media
Stasis Media specialises in creating accurate digital humans (Image credit: Sony / Peris Digital)
Sony has also worked closely with digital production agency Stasis Media, based in Plymouth, UK. Stasis Media specialises in digital human renders, where pixel-perfect images are needed to capture people that look truly real.
Each Stasis Media shoot now employs a number of Sony’s high-end Alpha 7R IV cameras to capture the intricate details of the face, and a handful of the smaller Sony RX0 II cameras to capture the body. All of the cameras are controlled and triggered via software to ensure that firing is virtually synchronous, and match the movements of the turntable, or the flaring of the LED lights. This is critical to enable the later process of accurately stitching the images together.
In the world of 3D scanning, one of the biggest challenges facing teams at the cutting edge is time
Craig Mason, director at Stasis Media says: “When you're trying to do a multi-camera shoot, it's really important that you manage to get the flashes synchronised with the camera exposures. We've found, particularly with the Alpha 7R IVs, that the synchronisation between multiple cameras and the flash exposures was very, very tight."
He continues: "In the world of 3D scanning, one of the biggest challenges facing teams at the cutting edge is time. Says Mason: “We were recently in Hamburg, creating 360-degree renders to help people learn how to cut and colour hair. The shoot was being undertaken in tandem with a very high-pressured video and photo collection being run at the same time, so we had just 60 to 120 seconds."
Mason adds: “When you've got a lot of cameras, it's very important to be able to coordinate the camera settings through an SDK like Sony’s.”
3D scanning: Lidar Lounge
Lidar Lounge creates scans of objects and people for companies including Warner Bros, Disney and Netflix (Image credit: Sony / Lidar Lounge)
One final customer of Sony Digital Imaging in photogrammetry has been the UK-based Lidar Lounge. Since 2014, Lidar Lounge has been in the business of creating digital humans, but have a strong track record as well in set and location scanning and vehicle and object scanning. The studio works for film and TV, gaming and virtual reality, and architecture and design clients.
The outfit offers a highly bespoke service encompassing all aspects of visual FX and scanning. Their client roster is impressive, ranging from large film studios (including Warner Bros, Disney, Universal and Netflix) through British institutions such as the British Museum, the National Portrait Gallery and the National Theatre, and on to large consumer brands that include Dyson, Intel and AEG.
Lidar Lounge’s fruitful relationship with Sony is likely to continue well into the future
Lidar Lounge have scanned a myriad of items in the British museum, after hours, including Egyptian mummies and the iconic Rosetta Stone. Normally these sorts of exhibits are only on view to the public behind glass protection, but the company was allowed to see them privately, up-close in order to create immaculate 3D scans of them.
Lidar Lounge’s fruitful relationship with Sony is likely to continue well into the future, with numerous innovations and new techniques expected to be possible with what Sony provide. In the short term, the company will begin to incorporate Sony’s pixel shift functionality into their light passes. They’re also looking to use Sony’s Camera Remote SDK to create more precise trigger timings with multiple lighting setups that vary based on the cameras position within their rigs.
This article was originally published in 3D World Magazine. Read more excellent 3D content in 3D World, the world's best-selling magazine for CG artists. Subscribe to 3D World.
Rob Redman is the editor of ImagineFX magazines and former editor of 3D World magazine. Rob has a background in animation, visual effects, and photography.
