The Digital Cinema Society has an interesting article on the question of whether compositing (and those who do it) are still needed at all when many classic compositing tasks are eliminated in the course of virtual film production. And what problems the new technology (still) brings, given ? the many advantages.
The technology of virtual film production was used on a large scale for Disney&s SF/Western series "The Mandalorian", set in the Star Wars universe, with the help of the company&s own effects forge ILM (Industrial Light and Magic) - around 50% of the production took place in front of the virtual film set.
Virtual film set of "The Mandalorian
It combines new techniques to merge the different stages of the traditionally linear process of producing a film (pre-production, shooting, post-production) using live computer-generated images on set to make them more intuitive. Its advantages are that the final product, the final look of a film (including grading and CGI images) is already visible to the film team during production and not only in the post-production phase. This makes film production faster, more collaborative, and gives immediate feedback to all involved on visual change requests.
Epic Games& Unreal Engine, a graphics engine and associated tools originally developed for creating and rendering artificial worlds in computer games, plays a central role. The degree of realism is so high with the current version that it can realistically depict many scenes in real time, thus providing the basis for virtual film production.
Using the virtual film set, the director and his team can scout or change the desired locations and determine camera settings and movements even before the shoot. Instead of green screens, four LED walls (3 sides plus ceiling) in the studio display the rendered film set, which can be modified in real time, e.g. to change the lighting or manipulate objects in it. This way, creative decisions for the final image can be made directly on set together by the participants of all departments (such as DOP, VFX, lighting and direction) and tested live together.
Virtual film set
So there are a lot of facts in favor of using LED walls instead of traditional green screens - but what are the disadvantages? What work has to be done by compositing after all?
The reality here is also more complex than at first sight, because virtual film production brings some improvements to classic workflows, but it also poses new challenges.
First, there is the problem that moiré can appear in the camera image if the LEDs of the background are similar in size to the pixels (or sensels) of the camera - checking the image on a control monitor can show whether the problem exists. If so, it can be fixed by blurring the background a bit or replacing the background in post-production.
Another problem can arise from the special light spectrum of RGB LEDs, which, unlike daylight or dedicated lights, is irregular and has peaks in narrow frequency bands - this can lead to unexpected color effects, especially when the LED light is combined with spotlights, which then have to be corrected in post-production.
Other compositing tasks include removing the distinct transition between the floor and the virtual background or correcting light reflections between the LED wall and the props. In post-production, the latency of the rendered background image must also be compensated for, which occurs when the camera makes a quick pan, for example, but the Unreal Engine only plays out the corresponding high-resolution background images with an offset of several frames (in "The Mandalorian", for example, the latency was 8 frames). more infos at bei www.digitalcinemasociety.org