Logo Logo
deutsch
directory schraeg
Knowledge
Codecs
Hardware
Camcorders· Cinema-Kamera· Computers· Drohnen· GPU· Kamera-Zubehör· Video-DSLR· accessories
Software
Compositing· Color correction· DV Editing
DV-Movies

HowTo
Shooting· Sound
Misc
Forschung· Reviews· Streaming
/// News
Perfect for music videos: Spatial object-based video editing via AI and 16 cameras

Perfect for music videos: Spatial object-based video editing via AI and 16 cameras

[11:01 sun,19.September 2021   by ]    

A particularly interesting field for the use of neural networks is the interpolation of perspective views from a few images (view synthesis) by means of so-called NeRFs (Neural layered Radiance Representaiton) - the trick is to calculate further photorealistic views from as few such images of an object as possible, which in some cases already works surprisingly well - even for images with shadows or reflections.





At the end of last year, a team from Google and the University of Oxford had already developed an www.slashcam.de/news/single/KI-ermoeglicht-automatisches-Layering-von-bewegten--16222.html interesting new AI algorithm for manipulating moving people in videos ), which recognizes different moving objects in a video and assigns them to different layers ,making a variety of interesting effects possible by compositing the individual layers in relation to each other.

Multiview
Images of the 16 Kamwras


The approach of the ShangHaiTech University team takes a different approach and tries to open up new possibilities of object-based spatial? video editing based on a large number of simultaneous views by using 16 cameras with a 180° field of view. video editing.

Multiview-dancer
Setup of the 16 cameras


And indeed - the new algorithm ("space-time coherent neural layered radiance representation" aka ST-NeRF) is not only able to separate different moving objects in the foreground from the background in videos, it also reconstructs correctly occluded backgrounds, allows cloning, resizing and re-timing of moving objects. All this also by virtual camera from different perspectives, which can be changed smoothly by virtual dolly ride.



The possibilities of the process for music videos, for example, are demonstrated in demo clips by the researchers - for example, the dance performance of two dancers can be perfectly synchronized afterwards, or musicians can float across the stage. Another application are virtual tracking shots that can be synthesized from the perspectives of the individual cameras at the director&s discretion, or a bullet-time effect.

Multiview-dancer
Duplicated dancers


Link more infos at bei jiakai-zhang.github.io

deutsche Version dieser Seite: Perfekt für Musikvideos: Räumliches, objektbasiertes Videoediting per KI und 16 Kameras

  

  Vorige News lesen Nächste News lesen 
bildDie Dark Bay: Größtes Virtual Production Studio Europas in Babelsberg bildRoland V-02HD MK II Streaming Video Mixer vorgestellt


related news:1E0fylm.ai: Professional AI-supported color grading via browser 16.October 2021
AI learns to fight - and synthesizes new martial arts moves 10.October 2021
AI separates music into separate vocal and accompaniment audio tracks 20.September 2021
Nvidia Broadcast App 1.3: Better live streaming and video conferencing via AI 6.September 2021
Omnimatte: Nearly perfect masks of moving objects via AI 27.August 2021
Per DeepFake itself star in the movie trailer of 22.August 2021
Alias-Free GAN: Nvidia's AI creates even more realistic animations of faces 27.July 2021
alle Newsmeldungen zum Thema Machine Learning
1E0DJI Ronin 4D gimbal and Zenmuse X9 full-frame 6K/8K camera: professional 4-axis stabilization 20.October 2021
Apple: Final Cut Pro 10.6 and Logic Pro 10.7 with new features and M1 Pro, M1 Max update 19.October 2021
fylm.ai: Professional AI-supported color grading via browser 16.October 2021
New Adobe Premiere Pro beta brings automatic color correction via Auto Tone 9.October 2021
Panasonic BS1H - cube cam with full-frame sensor under 600 grams 7.October 2021
Leitz ELSIE: 13 new full-frame Cine Prime lenses for 19,000 euros each 29.September 2021
Sirui 50mm T2.9 1.6x -- affordable anamorphic lens for full-frame soon on Indiegogo 27.September 2021
alle Newsmeldungen zum Thema DV Editing
1E0Sony C3 Portal: Cloud solution for Sony cameras - from shooting directly to the editing suite 23.September 2021
Adobe acquires Frame.io to strenghten collaborative editing with Premiere Pro 21.August 2021
Magix Vegas 19 available - including BRAW Editing and Text-to-Speech 19.August 2021
Audio Design Desk 1.7: Real-time editing tool for SFX now with DAW synchronization 18.July 2021
Postproduktion in der Cloud mit Adobe Premiere Pro per Qumulo Studio Q 5.June 2021
Change age, gender or facial expression of a face via AI Photo Editing 4.April 2021
Nikon: New, free image and video editing software NX-Studio incl. metadata support 4.March 2021
alle Newsmeldungen zum Thema Montage


[nach oben]
















Archiv Newsmeldungen

2021

October - September - August - July - June - May - April - March - February - January

2020
December - November - October - September - August - July - June - May - April - March - February - January

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

2001

2000






































deutsche Version dieser Seite: Perfekt für Musikvideos: Räumliches, objektbasiertes Videoediting per KI und 16 Kameras



last update : 24.Oktober 2021 - 15:02 - slashCAM is a project by channelunit GmbH- mail : slashcam@--antispam:7465--slashcam.de - deutsche Version