Virtual film production provides a good example of exciting high-end technology and new workflows that are first used by expensive Hollywood high-tech productions and then become more affordable over time and available in less complex productions.
Virtual film production combines new techniques to merge the different stages of the traditionally linear production process of a film (pre-production, shooting, post-production) with live computer-generated images on the set. Their advantages lie in the fact that the final product, the final look of a film (including grading and CGI images) is already visible to the film team during production and not only in the post-production phase. This makes film production faster, more collaborative, and provides instant feedback to all participants on visual change requests.
Virtual set with LED wall
The Unreal Engine by Epic Games plays a central role in the examples presented, a graphics engine and associated tools that were originally created to create and represent artificial worlds in computer games. With the current version 4.22, the degree of realism is so high that - assuming sufficient GPU power - it can realistically display many film scenes in real time and thus provide the basis for visual film production.
Using a virtual film set, the director and his team can scout or change the desired locations before shooting and determine camera settings and movements. Instead of green screens, large LED walls display the virtual film set, which can be changed in real time, e.g. to change the lighting or manipulate objects in it.
Virtual set with greenscreen
By tracking the camera movements, the perspective of the images changes interactively three-dimensionally and correctly, so that the set really looks like a real room or landscape when looking through the viewfinder. The lighting of the virtual set (i.e. entire landscapes) and the studio spotlights can also be changed synchronously in order to try out new looks in real time - a step that is normally only done much later in post-production.
This way, creative decisions for the final image can be made directly on the set by the participants of all departments (such as DOP, VFX, lighting and direction) and tested live together. Not quite as high-tech, the production can also take place in front of a classic green screen, but can then be watched by keying on the monitor in a virtual set rendered in real time.
Epic Games provides the Virtual Production Field Guide as a 98-page PDF for free download, explaining how virtual film production works and presenting projects that have already used it. It also demonstrates virtual production workflows for the different film departments. This is complemented by interviews with professionals - including director and actor Sir Kenneth Branagh, Oscar winner and VFX supervisor Ben Grossman and cameraman Bill Pope - who explain how they have used VP workflows in their productions. There is also Virtual Production Hub, a site where you can find many articles and podcasts on the topic.