Logo Logo
deutsch
directory schraeg
Knowledge
Codecs
Hardware
Camcorders· Cinema-Kamera· Computers· Drohnen· GPU· Kamera-Zubehör· Video-DSLR· accessories
Software
Compositing· Color correction· DV Editing
DV-Movies

HowTo
Shooting· Sound
Misc
Forschung· Reviews· Streaming
/// News
Disney combines CGI with AI for movie-like facial animations

Disney combines CGI with AI for movie-like facial animations

[11:00 Sun,5.December 2021   by ]    

We&ve already reported on Nvidia&s Style GAN deep learning algorithm for generating and animating and also on Disney&s experiments with DeepFake algorithms for animating virtual faces. Now Disney&s AI team is working on a special combination of these methods with classical CGI techniques to be able to simulate artificial faces in movie quality.


Disney-GAN-Faces-CGI-to-GAN
Combination of rendered face with GANStyle2


In this way, faces can now be digitized and simulated in very high quality by means of multi-camera recordings, but only the skin and the three-dimensional profile of the face, not the whole head - apart from the hair, the eyes and the interior of the mouth are missing in these models.

Disney-GAN-Faces-BW-Mask
CGI 3D face model without hair, eyes and mouth





Therefore, when using these face models in movies, a lot had to be retouched and supplemented by hand afterwards in an extremely elaborate way to get a realistic animated head, like for example in Star Wars the re-animation of the meanwhile deceased actors Peter Cushing (Grand Moff Tarkin) or Carrie Fisher (Princess Leia) in short scenes.

Disney-GAN-Faces-CGI-GAN
Combination of rendered face with GANStyle2


The new project now seeks to automate this final step via AI to combine high quality CGI facial renderings with the strengths of Deep Learning generated faces using github. com/NVlabs/stylegan2 (StyleGAN2) to combine them into an extremely real-looking virtual face - the latter have so far still suffered from the relatively low resolution (currently 1024×1024) of the faces generated by them, which is still far below that required for rendering in movies, and a roft lack of coherence across sequences of images, but shone in contrast to the CGI faces with their representation of hair, eyes and mouth.

Disney-GAN-Faces-wo-Mouth
rendered faces compared with GANStyle2 generated faces.


For this purpose, the skin surface generated by ray tracing is added to the face generated by StyleGAN2, including the head, by compositing. The new method also allows a realistic change of the lighting of the generated faces including eyes , hair and mouth with correct shadow casting:

Disney-GAN-Lighting-Schatten
Combination of rendered face with GANStyle2 with different light incidence.


Also more complex changes in ambient lighting in terms of direction and color - the AI algorithm adds eyes, hair to match the rendered face:

Disney-GAN-Lighting-komplex
Combination of rendered face with GANStyle2 with different light situation.


And, very important for the task Disney is targeting: the results also look realistic when animated, for example when changing from one facial expression to another or when speaking. Even (slight) changes in camera perspective can be made without problems, but with larger changes in perspective the algorithm still tends to create clearly visible artifacts. The method is not yet perfect, but it is a big step on the way to artificial faces suitable for film.

Link more infos at bei studios.disneyresearch.com

deutsche Version dieser Seite: Disney kombiniert CGI mit KI für filmtaugliche Gesichtsanimationen

  

  Vorige News lesen Nächste News lesen 
bildAdobe zeigt KI-Tools zur Mimik-Veränderung in Videos - Project Morpheus bildKostenlose Motion Cam App ermöglicht erstmals CinemaDNG RAW-Videoaufnahme auf Smartphones


related news:1E0New AI automatically perfectly exposes objects from photos 7.August 2022
Film simulation tool Dehancer Film Pro now also available as beta for Final Cut Pro 2.August 2022
Sony Future Filmmaker competition entices with visit to Sony Pictures film studios in Hollywood 1.August 2022
Large online auction of over 2,000 pieces of professional film equipment: ARRI, Canon, RED, ZEISS... 28.July 2022
Cinematic HFR: Titanic and Avatar get 25.July 2022
How Everything Everywhere All At Once was filmed 24.July 2022
Netflix removes movie noise, saves 30% bandwidth and adds it back again 21.July 2022
alle Newsmeldungen zum Thema Film
1E0New AI automatically perfectly exposes objects from photos 7.August 2022
AI-based grading solution fylm.ai v1.3 learns some context 26.July 2022
Anyone can do art: Meta's new AI acts as a painter at will 19.July 2022
Video stabilization: AI-supported Mercalli V6 SAL introduced 18.July 2022
Is AI Revolutionizing Photography? Object-based editing with the help of DALL-E 2 30.June 2022
DaVinci Resolve 18 Beta 5 - incl. AI stabilizer for gyro metadata and 10 bit preview 25.June 2022
See in the dark by AI with color 22.May 2022
alle Newsmeldungen zum Thema Machine Learning
1E0New option makes it easier to edit longer videos for YouTube Shorts 8.August 2022
Sony FX3 gets Netflix certification 3.August 2022
Netflix removes movie noise, saves 30% bandwidth and adds it back again 21.July 2022
Amazon Prime Day deals 2022 for filmmakers 12.July 2022
How does Netflix choose its cameras? A look behind the scenes 4.July 2022
Panasonic BS1H box camera receives Netflix approval 21.March 2022
Disney develops modular technology for cinema-sized holographic movies 20.March 2022
alle Newsmeldungen zum Thema Streaming Portale


[nach oben]
















Archiv Newsmeldungen

2022

August - July - June - May - April - March - February - January

2021
December - November - October - September - August - July - June - May - April - March - February - January

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

2001

2000






































deutsche Version dieser Seite: Disney kombiniert CGI mit KI für filmtaugliche Gesichtsanimationen



last update : 10.August 2022 - 18:02 - slashCAM is a project by channelunit GmbH- mail : slashcam@--antispam:7465--slashcam.de - deutsche Version