Logo Logo
deutsch
directory schraeg
Knowledge
Codecs
Hardware
Camcorders· Cinema-Kamera· Computers· Drohnen· GPU· Kamera-Zubehör· Video-DSLR· accessories
Software
Compositing· Color correction· DV Editing
DV-Movies

HowTo
Shooting· Sound
Misc
Forschung· Reviews· Streaming
/// News
Perfect for music videos: Spatial object-based video editing via AI and 16 cameras

Perfect for music videos: Spatial object-based video editing via AI and 16 cameras

[11:01 sun,19.September 2021   by ]    

A particularly interesting field for the use of neural networks is the interpolation of perspective views from a few images (view synthesis) by means of so-called NeRFs (Neural layered Radiance Representaiton) - the trick is to calculate further photorealistic views from as few such images of an object as possible, which in some cases already works surprisingly well - even for images with shadows or reflections.





At the end of last year, a team from Google and the University of Oxford had already developed an www.slashcam.de/news/single/KI-ermoeglicht-automatisches-Layering-von-bewegten--16222.html interesting new AI algorithm for manipulating moving people in videos ), which recognizes different moving objects in a video and assigns them to different layers ,making a variety of interesting effects possible by compositing the individual layers in relation to each other.

Multiview
Images of the 16 Kamwras


The approach of the ShangHaiTech University team takes a different approach and tries to open up new possibilities of object-based spatial? video editing based on a large number of simultaneous views by using 16 cameras with a 180° field of view. video editing.

Multiview-dancer
Setup of the 16 cameras


And indeed - the new algorithm ("space-time coherent neural layered radiance representation" aka ST-NeRF) is not only able to separate different moving objects in the foreground from the background in videos, it also reconstructs correctly occluded backgrounds, allows cloning, resizing and re-timing of moving objects. All this also by virtual camera from different perspectives, which can be changed smoothly by virtual dolly ride.



The possibilities of the process for music videos, for example, are demonstrated in demo clips by the researchers - for example, the dance performance of two dancers can be perfectly synchronized afterwards, or musicians can float across the stage. Another application are virtual tracking shots that can be synthesized from the perspectives of the individual cameras at the director&s discretion, or a bullet-time effect.

Multiview-dancer
Duplicated dancers


Link more infos at bei jiakai-zhang.github.io

deutsche Version dieser Seite: Perfekt für Musikvideos: Räumliches, objektbasiertes Videoediting per KI und 16 Kameras

  

  Vorige News lesen Nächste News lesen 
bildDie Dark Bay: Größtes Virtual Production Studio Europas in Babelsberg bildRoland V-02HD MK II Streaming Video Mixer vorgestellt


related news:1E0Illuminated only by starlight: New AI algorithm denoises videos perfectly 16.May 2022
NAB 2022 Tutorialclip: DaVinci Resolve 18 - AI-Mask Objects, Depth Mask, Surface Tracker 27.April 2022
AI helps to synchronize motion pictures 16.April 2022
Apple Final Cut Pro update brings duplicate detection and voice isolation via AI 13.April 2022
DALL-E 2: AI generates and edits images based on text description only 10.April 2022
AI generates frighteningly accurate portraits - based on voice alone 9.April 2022
New AJA Ki Pro GO firmware 4.0 brings new playback options 3.April 2022
alle Newsmeldungen zum Thema Machine Learning
1E0Magix Video Pro X14 available with new "high-speed" timeline 20.May 2022
Adobe Premiere Pro Update 22.4 brings 10x faster 4:2:0 HEVC export 12.May 2022
Open source video editor Olive 0.2 said to be nearing completion 9.May 2022
NAB 2022 Interview: Camera to Cloud, Frame IO and Premiere Pro 2.May 2022
NAB 2022 Tutorialclip/Interview: How does Frame.IO work in Premiere Pro? 28.April 2022
Camera app FiLMiC Pro gets support for Frame.io Camera-to-Cloud 22.April 2022
Apple Final Cut Pro update brings duplicate detection and voice isolation via AI 13.April 2022
alle Newsmeldungen zum Thema DV Editing
1E0Video editing via AI: Simply replace backgrounds or make objects disappear 21.January 2022
Audio Design Desk 1.9: Real-time editing tool for sound effects now with new features and Lite versi 15.January 2022
Hedge Postlab for Media Composer enables team editing in the cloud for Avid editors 6.November 2021
Sony C3 Portal: Cloud solution for Sony cameras - from shooting directly to the editing suite 23.September 2021
Adobe acquires Frame.io to strenghten collaborative editing with Premiere Pro 21.August 2021
Magix Vegas 19 available - including BRAW Editing and Text-to-Speech 19.August 2021
Audio Design Desk 1.7: Real-time editing tool for SFX now with DAW synchronization 18.July 2021
alle Newsmeldungen zum Thema Montage


[nach oben]
















Archiv Newsmeldungen

2022

May - April - March - February - January

2021
December - November - October - September - August - July - June - May - April - March - February - January

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

2001

2000






































deutsche Version dieser Seite: Perfekt für Musikvideos: Räumliches, objektbasiertes Videoediting per KI und 16 Kameras



last update : 21.Mai 2022 - 15:02 - slashCAM is a project by channelunit GmbH- mail : slashcam@--antispam:7465--slashcam.de - deutsche Version