For the better part of the 21st century, visual effects in film and television have revolved around the arduous process of compositing computer-generated images onto footage filmed using green screens. Yet, despite this process becoming more ubiquitous and technological improvements, the process still has its limitations. For years now some audiences have grown tired of poorly done effects that clearly look like an artificial background composited onto live footage in post-production. And the reason for this sterile, artificial look is often due to one key limitation – lighting.
Digital effects work takes an obscene amount of time and effort, but the greatest struggle is producing lighting which appears to naturally interact with the live-action elements of a shot. And the fact that digital effects are mostly handled in post-production after principal photography has wrapped makes it an especially challenging problem to work around.
Seemingly no one was more frustrated with this limitation than director Jon Favreau, who attempted to work his way around it while directing the remakes of The Jungle Book and The Lion King. These were shot exclusively on blue and green screens, and Favreau’s solution to making the lighting appear more natural was through the use of large LED screens around characters to create interactive lighting – an easy fix, right? Well, Favreau characterised the actual process of constructing and adjusting this interactive lighting as incredibly labour intensive.
That’s where The Mandalorian comes in. While producing this show, Favreau used cutting edge technology developed in partnership with Industrial Light & Magic (the pioneering visual effects company founded by George Lucas for the original Star Wars trilogy) and Epic Games. These developers utilised Epic’s Unreal Engine to painstakingly create photorealistic backgrounds which are then displayed on LED screens. A 21 ft tall circular set covered in LED panels known as ‘The Volume’ then displays this ‘video wall’ of completed computer-generated backgrounds.
The video wall provides interactive lighting and the physical set is constructed to fit with the images produced. The actors aren’t looking at empty green-screens that will be filled in post-production, they’re looking at completed effects. Instead of imagining a mountain, the mountain is right in front of them. This also solves the problem of effects artists having to work around already completed footage; if there’s something wrong with the shot, a team of effects artists is on hand to adjust the images in real-time.
The process isn’t flawless however, there still has to be some slight colour correction to match the physical set with the video wall, and camera sensors and lenses must be adjusted to avoid the ‘Moiré effect’ that accompanies filming a screen. While the costs of producing a set such as the Volume was likely considerable, the amount of hours saved of compositing CGI around completed footage is likely a relief to digital effects artists, who now can adjust their work during production to produce the highest quality shot.
The implications of this new technology are exciting, and how future filmmakers and effects artists will utilise it will be interesting to see. However, the cost of producing these types of sets means the technology will be restricted to only the studios with the deepest pockets and will likely be some years before it can become the industry standard. That’s the redeeming feature of green screens, they’ve been around since before the digital age and they’re affordable to just about everyone. Regardless of how soon to expect further developments in this technology, it really is a marvel to look back and see what it’s been able to accomplish in The Mandalorian.
