Now Disney is also involved in researching deep-fake technology, which is logical, given its enormous potential for film production. However, the prerequisite for professional use is a better quality than before. Were the many clips with DeepFakes mainly demonstrations that proved that the technology works, but often no great emphasis was placed on quality, which produces appealing / photorealistic results even at high resolutions.
Disney, together with ETH Zurich, has now developed a method to exchange faces in videos and images fully automatically via a neural network. This means that the face of an actor can be exchanged with that of any person and thus be animated by them in a deceptively realistic way.
According to Disney, this is the first algorithm that is able to deliver results in megapixel quality that are also temporally coherent (i.e. that there are no strong changes between some of the frames of a video, as is often the case). And indeed, the results look very good already.
The video explains the individual steps of the face exchange method and demonstrates them using different people:
VY=yji0t6KS7Qo
At the end of the video the new method is compared with several other DeepFake algorithms - it scores significantly better than all other previous methods. Increasing the resolution to now 1,024x1,024 pixels is a big leap - until now, DeepFake videos often had a resolution of only 256x256 pixels.
Disney's DeepFakes compared to other methods
For example, the new method not only adjusts the contrast of the new face to the respective exposure situation, but also avoids too large unwanted differences in facial expression between individual images.
The interest in DeepFakes by a large film studio such as Disney is logical - after all, in the past re-animation of dead actors such as Peter Cushing (Grand Moff Tarkin) or Carrie Fisher (Princess Leia) for Star Wars films, complex 3D models had to be used to produce very time-consuming film scenes of a few seconds in length. With the help of a sophisticated DeepFake algorithm, such tasks would be much easier to perform and would save a lot of money during production - not to mention the many other possibilities of using DeepFakes in movies.
A challenge for the future are of course videos in which the faces of the actors do not look into the camera head-on.
;)
;)
;)