Nothing less than democratizing the use of virtual, computer-generated characters in film production by simplifying and cheapening the steps required to do so is the big goal of startup Wonder Dynamic. And indeed, the demos of Wonder Studio, which has been three years in development, look amazing: the movements of an actor in a movie scene are automatically analyzed and can then be transferred to any 3D character, which is animated as a result.
CG character integrated into the scene.
This virtual character then seamlessly replaces the actor, mimics both his movements and facial expressions, and is also properly lit and adapted to the film look to match the scene - the whole thing works not only for static but also for dynamic camera settings including changing lighting. In contrast to other solutions, this does not require any special depth information, two cameras (as for example with Move.ai) or markers - the recording of a single normal camera is sufficient for motion capturing, and the AI does the rest.
CG figure selection
Wonder Studio can work not only with single shots, but also with entire movie sequences. It automatically detects cuts and tracks each actor through the whole scene across all the different shots, replacing them with the selected CG model. It can also replace multiple actors with different CG characters. Some ready-made models are already included as defaults in Wonder Studio and can be used at no additional cost, but you can also import any of your own CG characters.
Wonder Studio thus automates a whole series of normally very laborious steps in post-production - starting with motion and face capturing, through the animation of body, hands and face, to the replacement of the figure in the scene by compositing including correct lighting and automatic addition of the background if the virtual figure is smaller than the actor serving as a model for the animation. The size of the virtual character in the scene can be scaled as desired and the movements edited. Wonder Studio can be used as easily by amateurs as it can be integrated into existing professional VFX workflows - it imports 3D models from popular modelers and exports data such as motion capture, camera tracking, the alpha mask or blank scene (without characters) for work in other programs. The finished scene can be rendered in up to 4K as an MP4 or PNG sequence (and soon ProRes).
Due to the high complexity and associated cost of this work, the use of such virtual characters in real film was previously reserved for large productions such as Disney&s Avenger series of blockbusters. For Wonder Studio, a camera is now all that is needed as hardware. Since Wonder Studio is a web-based service, it even eliminates the need for a powerful computer for 3D animation or compositing. The goal of Wonder Studio is to free creative work from the mechanical and rather tedious routine tasks by automating them via AI. While various AI tools already exist for individual tasks (such as motion capture or background replacement), Wonder Studio offers seamless integration of these diverse tools.
The whole thing sounds too good to be true, but it is already confirmed by first reports like for example from Techcrunch, that it really works as good as advertised. While Wonder Studio is still in closed beta and you have to apply for access, it is already being used by well-known action directors Russo Brothers for the production of a new Netflix movie starring Millie Bobby Brown and Chris Pratt.
There should soon be a way to try Wonder Studio for free after the closed beta - advanced features for professionals will be available in a variety of paid plans. Wonder Dynamics didn&t come out of nowhere either - one of its founders is well-known actor Tye Sheridan, who starred in Steven Spielberg&s SF film Ready Player One (2018). By the end of 2021, Epic Games and Samsung, among others, had invested million in the startup.