Logo Logo
deutsch
directory schraeg
Knowledge
Codecs
Hardware
Camcorders· Cinema-Kamera· Computers· Drohnen· GPU· Kamera-Zubehör· Video-DSLR· accessories
Software
Compositing· Color correction· DV Editing
DV-Movies

HowTo
Shooting· Sound
Misc
Forschung· KI· Reviews· Streaming
/// News
Perfect for music videos: Spatial object-based video editing via AI and 16 cameras

Perfect for music videos: Spatial object-based video editing via AI and 16 cameras

[11:01 Sun,19.September 2021   by ]    

A particularly interesting field for the use of neural networks is the interpolation of perspective views from a few images (view synthesis) by means of so-called NeRFs (Neural layered Radiance Representaiton) - the trick is to calculate further photorealistic views from as few such images of an object as possible, which in some cases already works surprisingly well - even for images with shadows or reflections.





At the end of last year, a team from Google and the University of Oxford had already developed an www.slashcam.de/news/single/KI-ermoeglicht-automatisches-Layering-von-bewegten--16222.html interesting new AI algorithm for manipulating moving people in videos ), which recognizes different moving objects in a video and assigns them to different layers ,making a variety of interesting effects possible by compositing the individual layers in relation to each other.

Multiview
Images of the 16 Kamwras


The approach of the ShangHaiTech University team takes a different approach and tries to open up new possibilities of object-based spatial? video editing based on a large number of simultaneous views by using 16 cameras with a 180° field of view. video editing.

Multiview-dancer
Setup of the 16 cameras


And indeed - the new algorithm ("space-time coherent neural layered radiance representation" aka ST-NeRF) is not only able to separate different moving objects in the foreground from the background in videos, it also reconstructs correctly occluded backgrounds, allows cloning, resizing and re-timing of moving objects. All this also by virtual camera from different perspectives, which can be changed smoothly by virtual dolly ride.



The possibilities of the process for music videos, for example, are demonstrated in demo clips by the researchers - for example, the dance performance of two dancers can be perfectly synchronized afterwards, or musicians can float across the stage. Another application are virtual tracking shots that can be synthesized from the perspectives of the individual cameras at the director&s discretion, or a bullet-time effect.

Multiview-dancer
Duplicated dancers


Link more infos at bei jiakai-zhang.github.io

deutsche Version dieser Seite: Perfekt für Musikvideos: Räumliches, objektbasiertes Videoediting per KI und 16 Kameras

  

  Vorige News lesen Nächste News lesen 
bildDie Dark Bay: Größtes Virtual Production Studio Europas in Babelsberg bildRoland V-02HD MK II Streaming Video Mixer vorgestellt


related news:1E0Google shows AI-bulltet-time effect incl. slow motion, blur and stabilizer - DynIBaR 30.September 2023
HeyGen Video Translate offers easy translation and dubbing of video clips 28.September 2023
Google Pixel 8 (Pro) - New level of "computational" videography through AI? 25.September 2023
YouTube: new editing app available - and video AI tools coming soon 22.September 2023
DALL-E 3 is here and generates more accurate AI images thanks to ChatGPT integration 21.September 2023
Video clip - Adobe Premiere Pro explained: Automatically remove pauses and filler words with AI, etc. 19.September 2023
Adobe introduces credit-based model for generative AI (and Creative Cloud gets more expensive) 18.September 2023
alle Newsmeldungen zum Thema KI
1E0Last Call - Final Spurt! The Nikon RAW Grading Contest ends on September 27th 2023 at 21h! 25.September 2023
YouTube: new editing app available - and video AI tools coming soon 22.September 2023
Nikon Z f retro-look full-frame DSLM (FM2) unveiled for 2,499 Euro: 24MP sensor, 4K 10-bit 60p ... 20.September 2023
Video clip - Adobe Premiere Pro explained: Automatically remove pauses and filler words with AI, etc. 19.September 2023
Atomos Edit - new extension for Adobe Premiere Pro 17.September 2023
Blackmagic Cinema Camera 6K introduced - full frame sensor and L-mount for 2.805 Euros 14.September 2023
Premiere Pro, After Effects and Frame.io - More AI, 3D Workspace and AWS Cloud 13.September 2023
alle Newsmeldungen zum Thema DV Editing
1E0YouTube: new editing app available - and video AI tools coming soon 22.September 2023
Atomos Edit Public Beta available - cloud-based video editing for fastest possible playout 27.June 2023
Node-based editing for After Effects - Hollywood Illusion launches Magic Nodes 1.5 31.May 2023
Atomos Edit introduced: Online Editing Coupled with Connect/Coud Capabilities 28.April 2023
Asus ROG Ally: Gaming handheld as an ultra-mobile editing system? 27.April 2023
Resolve 18.5: New features explained with Screencaptures: Relight, Text-Based Editing ... 17.April 2023
Atomos Edit: browser based video editing within Atomos Cloud Studio 14.April 2023
alle Newsmeldungen zum Thema Montage


[nach oben]
















Archiv Newsmeldungen

2023

October - September - August - July - June - May - April - March - February - January

2022
December - November - October - September - August - July - June - May - April - March - February - January

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

2001

2000






































deutsche Version dieser Seite: Perfekt für Musikvideos: Räumliches, objektbasiertes Videoediting per KI und 16 Kameras



last update : 1.Oktober 2023 - 18:02 - slashCAM is a project by channelunit GmbH- mail : slashcam@--antispam:7465--slashcam.de - deutsche Version