VideoGigaGAN - Adobe shows impressive Super Resolution process for moving images
[11:41 Thu,25.April 2024 by Rudi Schmidts]
Super resolution refers to algorithms that can artificially increase the resolution of an image - in other words, add more image information than was originally recorded by a camera. With the emergence of deep learning AI models, super resolution has also received an enormous qualitative boost in recent years. VideoGigaGAN from an Adobe research department now promises a new level of quality, which aims to solve some typical problems even better than current AI models.
The simple application of super resolution algorithms to moving images (Video Super Resolution, VSR) usually leads to artifacts such as flickering and temporal inconsistencies in patterns and motion sequences. However, in contrast to conventional VSR methods, Adobe's VideoGigaGAN should now be able to generate temporally consistent videos even with finely structured display details - but without the artifacts that were previously typical. The associated project website already presents astonishing results in an interactive before-and-after comparison:
In particular, Adobe's VideoGigaGAN seems to get the typical problem areas of temporal flickering and alias artifacts under control in a really impressive way. Typical upscaling problems such as “floating” textures in faces and other moving objects also do not occur anywhere in the demo video clips. The following clip shows a detailed example:
At the same time (hats off!) Adobe also points out two current weaknesses of its model: The model tends to artifact slightly after more than 200 frames and the font is also often not scaled correctly.
Font still causes problems with generative upscaling.
However, despite these weaknesses, Adobe's VideoGigaGAN already shows a path for future AI post-production. For example, synthetic/generative or real production can be carried out with significantly reduced resolution and the details are only then “conclusively added” in post-production. The bottom line is that you don't have to “shoot as cleanly” - the AI in post-production provides the nice touch anyway. And all this without any post-production know-how.
;)
;)