[11:01 Sun,19.September 2021 by Thomas Richter] |
A particularly interesting field for the use of neural networks is the interpolation of perspective views from a few images (view synthesis) by means of so-called NeRFs (Neural layered Radiance Representaiton) - the trick is to calculate further photorealistic views from as few such images of an object as possible, which in some cases already works surprisingly well - even for images with shadows or reflections.
At the end of last year, a team from Google and the University of Oxford had already developed an www.slashcam.de/news/single/KI-ermoeglicht-automatisches-Layering-von-bewegten--16222.html interesting new AI algorithm for manipulating moving people in videos ), which recognizes different moving objects in a video and assigns them to different layers ,making a variety of interesting effects possible by compositing the individual layers in relation to each other. ;) Images of the 16 Kamwras The approach of the ShangHaiTech University team takes a different approach and tries to open up new possibilities of object-based spatial? video editing based on a large number of simultaneous views by using 16 cameras with a 180° field of view. video editing. ;) Setup of the 16 cameras And indeed - the new algorithm ("space-time coherent neural layered radiance representation" aka ST-NeRF) is not only able to separate different moving objects in the foreground from the background in videos, it also reconstructs correctly occluded backgrounds, allows cloning, resizing and re-timing of moving objects. All this also by virtual camera from different perspectives, which can be changed smoothly by virtual dolly ride. The possibilities of the process for music videos, for example, are demonstrated in demo clips by the researchers - for example, the dance performance of two dancers can be perfectly synchronized afterwards, or musicians can float across the stage. Another application are virtual tracking shots that can be synthesized from the perspectives of the individual cameras at the director&s discretion, or a bullet-time effect. ;) Duplicated dancers  more infos at bei jiakai-zhang.github.iodeutsche Version dieser Seite: Perfekt für Musikvideos: Räumliches, objektbasiertes Videoediting per KI und 16 Kameras |
 |
|
|