In the explosion of new technologies and techniques that have hit the film and television production industry in recent years, it is possible that the term ‘volumetric video’ has not yet achieved the general profile or understanding of – for example – VR or AR. But with an increase in the flow of volumetric projects from 2019 and some flagship productions expected to start in 2023, this will certainly not be the case for much longer.
What is volumetric video?
So first: what is volumetric video? Developed to bring depth and new creative possibilities to video production, this technique involves the use of multiple cameras to capture three-dimensional space. Typically, the type of content captured in this way may include a person, an object, or an entire performance. Afterwards, the video can then be replayed using a smart device, a 2D computer viewer, VR glasses or an AR display – allowing the viewer to see the object from any angle.
To date, the significant processing and general hardware requirements of volumetric video have meant that its reach has been somewhat limited. But as with every visual technology before it, the costs associated with capture and delivery are gradually falling as processing power increases and the surrounding product ecosystem becomes more comprehensive. Indeed, it is possible that a tipping point for this technique could be in sight as early as next year.
‘The publishing tool was the missing piece’
One of a handful of leading innovators currently focused on volumetric video solutions, Arcturus was founded by a creative and executive team that has worked at companies including YouTube, Google, Dreamworks and Pixar to their collective credit. The starting point for the business that became Arcturus in 2016 was: “the realization that one piece was missing, and that was a tool for post-production and volumetric video streaming. So that’s what they set out to build,” explained Piotr Uzarowicz, Head of Partnerships and Marketing at Arcturus.
Read more Accelerator – Volumetric video for broadcast, metaverse production and RT3D assets
The result of their work is HoloSuite, a SaaS post-production platform that does not depend on recording and offers tools for editing and distribution of volumetric video. “It consists of two parts: HoloEdit, which contains tools for editing and compressing the recording so that it can be sent to [immersive creation tools] Unreal or WebXR, and HoloStream, which is a streaming solution for mobile, web, desktop and immersive headsets.”
Arcturus has also consulted and undertaken production work on a number of high-profile projects, including Hulu’s Emmy-nominated Magic Leap ‘Light as a Feather’ activation and Madonna’s volumetric holograms at the 2019 Billboard Music Awards. The latter project – billed as the first-ever live volumetric performance – proved particularly significant in terms of conveying the long-term potential of volumetric performance to a wider audience, although it is clear that we should expect live volumetric productions to appear more frequently from here on out.
To that end, Uzarowicz suggests that—rather than preventing its development—the pandemic may have provided an opportunity for “more people to take the time to learn [the technique]. Another interesting aspect is that during the production of the volumetric video, the artist is alone in that space. You can really separate the components, so you can have a camera crew in one room, an actor in another room, and a director joining remotely on Zoom. [Volumetric video] he facilitated the accommodation [Covid] limitations.”
Meanwhile, HoloSuite tools continue to evolve, with a particular focus on simplifying the recording and distribution process. For example, a late 2021 update added full support for Scatter’s Depthkit Studio 3D capture software, allowing content creators to capture, edit and distribute their volumetric projects using a single pipeline, making it easier to create holograms for AR/VR, frame media and Spatial 360 degree videos.
“Volumetric and virtual production intersect”
The appearance at the Madonna/Billboard Awards was also a turning point for Dimension Studio, because it was in their London facility that the singer’s creative director Jamie King and Reflector Entertainment shot four of her ‘personas’ – namely, musician, cha-cha instructor, secret agent and bride. . Volumetrically captured models and effects were rendered live via Reflektor’s custom volumetric capture player, with high-quality playback taking place within Unreal Engine 4. The resulting AR performance – which also included special effects such as clouds, rain and butterflies – was presented alongside the live components , including Latin singer Maluma and more than a dozen dancers.
Surveying the current production landscape, Co-Founder and Co-CEO of Dimension Studio Simon Windsor noted the tendency for “virtual production and volumetric video work to become increasingly intertwined these days.” It will also come as no surprise to find that the sport has proven to be an early adopter, with comprehensive volumetric capture capabilities allowing for detailed analysis of player performance. For example, Dimension’s Polymotion Stage was used during the production of Sky Sports’ free-to-view swing analysis segment at the 148th Open, providing a full 360-degree view of a golfer’s swing. After the golfers were captured in the studio, their holograms were used on air just 48 hours later.
While sports and entertainment are undoubtedly leading the way in this field, Windsor says there are strong signs of “3D tech [being adopted] for everything from concept visualization to production and post-production.” He suggests that a significant increase in live volumetric recording is just around the corner, while one of Dimension’s current areas of research and development is using volumetric video to “create a digital audience, which leverages the experience and systems we’ve developed over the last few years. If you are able to record [a crowd] volumetrically, it gives you a lot more flexibility in terms of changing aspects such as clothing, hairstyle and so on.”
“It must fit volumetrically with the rest of the post”
4Dviews is another company for whom the scope of volumetric video has become increasingly expansive. Founded in 2007, the company’s product offering is now led by the HOLOSYS volumetric video recording system, which includes recording equipment (cameras, lighting and electronics), a software package with a lifetime product license, installation and operator and user training. With the inclusion of Unity3D and Unreal Engine plug-ins, the Alembic format (.ABC) export option allows users to natively import any volumetric video into Cinema4D, Autodesk Maya, Houdini, and Blender3D.
The starting point for any creative to start using volumetric video is to “adjust to the fact that video now has three dimensions, which may seem a bit strange at first, I guess!” said CEO Richard Broadbridge. Regardless, 4Dviews’ philosophy has always been to make the whole process as seamless as possible: “Our approach is that it has to fit in with the rest of the post-production process, whether it’s web stuff, 2D presentations or music videos.”
Indeed, music promotions have been a strong contributor to the client base since the very beginning, along with projects in the world of fashion and art. And as the market grew, so did the expectations regarding image quality: “The main focus since time immemorial has been on image quality, and there is no end to that.”
Like other contributors to this feature, Broadbridge points out that Covid-era lockdowns have resulted in more intense development work across the industry. “All of a sudden, people could focus more on research and development than on imaging activities,” he recalls, adding that in the case of 4Dviews, there was a chance for “our engineers to try to move forward as much as possible. [In terms of projects] we’ve had a very good year this year and I think the market as a whole feels more confident going forward; there’s certainly a lot of experimentation going on.”
In fact, a recent Research and Markets forecast suggests that the sky’s the limit for volumetric video, with the global market expected to grow from $1.5 billion in 2021 to $4.9 billion by 2026 – representing a CAGR of 26.9 %. As well as volumetrics’ ability to deliver enhanced sports and entertainment viewing experiences, Research and Markets cites factors such as growing demand for AR in retail and e-commerce and an overall growing level of investment in the AR/VR ecosystem.
In short, it appears that we are still in the relatively early chapters of what will become a major global success story.
Read more Artificial intelligence in broadcasting