Logo Logo
deutsch
directory schraeg
Knowledge
Codecs
Hardware
Camcorders· Cinema-Kamera· Computers· Drohnen· GPU· Kamera-Zubehör· Video-DSLR· accessories
Software
Compositing· Color correction· DV Editing
DV-Movies

HowTo
Shooting· Sound
Misc
Forschung· Reviews· Streaming
/// News
Disney combines CGI with AI for movie-like facial animations

Disney combines CGI with AI for movie-like facial animations

[11:00 sun,5.December 2021   by ]    

We&ve already reported on Nvidia&s Style GAN deep learning algorithm for generating and animating and also on Disney&s experiments with DeepFake algorithms for animating virtual faces. Now Disney&s AI team is working on a special combination of these methods with classical CGI techniques to be able to simulate artificial faces in movie quality.


Disney-GAN-Faces-CGI-to-GAN
Combination of rendered face with GANStyle2


In this way, faces can now be digitized and simulated in very high quality by means of multi-camera recordings, but only the skin and the three-dimensional profile of the face, not the whole head - apart from the hair, the eyes and the interior of the mouth are missing in these models.

Disney-GAN-Faces-BW-Mask
CGI 3D face model without hair, eyes and mouth





Therefore, when using these face models in movies, a lot had to be retouched and supplemented by hand afterwards in an extremely elaborate way to get a realistic animated head, like for example in Star Wars the re-animation of the meanwhile deceased actors Peter Cushing (Grand Moff Tarkin) or Carrie Fisher (Princess Leia) in short scenes.

Disney-GAN-Faces-CGI-GAN
Combination of rendered face with GANStyle2


The new project now seeks to automate this final step via AI to combine high quality CGI facial renderings with the strengths of Deep Learning generated faces using github. com/NVlabs/stylegan2 (StyleGAN2) to combine them into an extremely real-looking virtual face - the latter have so far still suffered from the relatively low resolution (currently 1024×1024) of the faces generated by them, which is still far below that required for rendering in movies, and a roft lack of coherence across sequences of images, but shone in contrast to the CGI faces with their representation of hair, eyes and mouth.

Disney-GAN-Faces-wo-Mouth
rendered faces compared with GANStyle2 generated faces.


For this purpose, the skin surface generated by ray tracing is added to the face generated by StyleGAN2, including the head, by compositing. The new method also allows a realistic change of the lighting of the generated faces including eyes , hair and mouth with correct shadow casting:

Disney-GAN-Lighting-Schatten
Combination of rendered face with GANStyle2 with different light incidence.


Also more complex changes in ambient lighting in terms of direction and color - the AI algorithm adds eyes, hair to match the rendered face:

Disney-GAN-Lighting-komplex
Combination of rendered face with GANStyle2 with different light situation.


And, very important for the task Disney is targeting: the results also look realistic when animated, for example when changing from one facial expression to another or when speaking. Even (slight) changes in camera perspective can be made without problems, but with larger changes in perspective the algorithm still tends to create clearly visible artifacts. The method is not yet perfect, but it is a big step on the way to artificial faces suitable for film.

Link more infos at bei studios.disneyresearch.com

deutsche Version dieser Seite: Disney kombiniert CGI mit KI für filmtaugliche Gesichtsanimationen

  

  Vorige News lesen Nächste News lesen 
bildAdobe zeigt KI-Tools zur Mimik-Veränderung in Videos - Project Morpheus bildKostenlose Motion Cam App ermöglicht erstmals CinemaDNG RAW-Videoaufnahme auf Smartphones


related news:1E0CAME-TV 5.4 24.January 2022
Video editing via AI: Simply replace backgrounds or make objects disappear 21.January 2022
Irix 21mm T1.5 - one more wide angle cine lens 13.January 2022
Micro LED, Neo QLED and (not yet) QD OLED - Samsung's TV technologies 2022 3.January 2022
Say again? Why some movie dialogs are so hard to understand 7.December 2021
AI calculates video clip from image sequence - ADOP 30.November 2021
Hedge Postlab for Media Composer enables team editing in the cloud for Avid editors 6.November 2021
alle Newsmeldungen zum Thema Film
1E0Video editing via AI: Simply replace backgrounds or make objects disappear 21.January 2022
HumanNeRF - AI models human bodies 14.January 2022
Try it out for yourself: This AI changes the age of faces 12.December 2021
Adobe shows AI tools to change facial expressions in videos - Project Morpheus 4.December 2021
Neural Nano Optics -With Metamaterial Optics and AI to the Salt Grain Camera 2.December 2021
Define video masks simply by description using new AI algorithm 1.December 2021
AI calculates video clip from image sequence - ADOP 30.November 2021
alle Newsmeldungen zum Thema Machine Learning
1E0Why color management? Netflix gives the answer 27.December 2021
Netflix: free webinars for aspiring filmmakers, writers and producers 12.December 2021
YouTube's Content ID in numbers - 722 million claims in 6 months 10.December 2021
YouTube rewards shorts with 100 million US dollars - also in Germany 4.October 2021
How to make a Netflix documentary about any topic 3.October 2021
Netflix opens office in Berlin and invests 500 million euros in German productions 17.September 2021
Vimeo now also supports Dolby Vision HDR 13.September 2021
alle Newsmeldungen zum Thema Streaming Portale


[nach oben]
















Archiv Newsmeldungen

2022

January

2021
December - November - October - September - August - July - June - May - April - March - February - January

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

2001

2000






































deutsche Version dieser Seite: Disney kombiniert CGI mit KI für filmtaugliche Gesichtsanimationen



last update : 28.Januar 2022 - 10:54 - slashCAM is a project by channelunit GmbH- mail : slashcam@--antispam:7465--slashcam.de - deutsche Version