Logo Logo
deutsch
directory schraeg
Knowledge
Codecs
Hardware
Camcorders· Cinema-Kamera· Computers· Drohnen· GPU· Kamera-Zubehör· Video-DSLR· accessories
Software
Compositing· Color correction· DV Editing
DV-Movies

HowTo
Shooting· Sound
Misc
Forschung· Reviews· Streaming
/// News
Perfect for music videos: Spatial object-based video editing via AI and 16 cameras

Perfect for music videos: Spatial object-based video editing via AI and 16 cameras

[11:01 Sun,19.September 2021   by ]    

A particularly interesting field for the use of neural networks is the interpolation of perspective views from a few images (view synthesis) by means of so-called NeRFs (Neural layered Radiance Representaiton) - the trick is to calculate further photorealistic views from as few such images of an object as possible, which in some cases already works surprisingly well - even for images with shadows or reflections.





At the end of last year, a team from Google and the University of Oxford had already developed an www.slashcam.de/news/single/KI-ermoeglicht-automatisches-Layering-von-bewegten--16222.html interesting new AI algorithm for manipulating moving people in videos ), which recognizes different moving objects in a video and assigns them to different layers ,making a variety of interesting effects possible by compositing the individual layers in relation to each other.

Multiview
Images of the 16 Kamwras


The approach of the ShangHaiTech University team takes a different approach and tries to open up new possibilities of object-based spatial? video editing based on a large number of simultaneous views by using 16 cameras with a 180° field of view. video editing.

Multiview-dancer
Setup of the 16 cameras


And indeed - the new algorithm ("space-time coherent neural layered radiance representation" aka ST-NeRF) is not only able to separate different moving objects in the foreground from the background in videos, it also reconstructs correctly occluded backgrounds, allows cloning, resizing and re-timing of moving objects. All this also by virtual camera from different perspectives, which can be changed smoothly by virtual dolly ride.



The possibilities of the process for music videos, for example, are demonstrated in demo clips by the researchers - for example, the dance performance of two dancers can be perfectly synchronized afterwards, or musicians can float across the stage. Another application are virtual tracking shots that can be synthesized from the perspectives of the individual cameras at the director&s discretion, or a bullet-time effect.

Multiview-dancer
Duplicated dancers


Link more infos at bei jiakai-zhang.github.io

deutsche Version dieser Seite: Perfekt für Musikvideos: Räumliches, objektbasiertes Videoediting per KI und 16 Kameras

  

  Vorige News lesen Nächste News lesen 
bildDie Dark Bay: Größtes Virtual Production Studio Europas in Babelsberg bildRoland V-02HD MK II Streaming Video Mixer vorgestellt


related news:1E0Heavy Metal meets Midjourney: Music video made from 10,000 AI-generated images 4.December 2022
Meta Encodec uses AI to compress audio files significantly more than MP3 21.November 2022
AI copies movie style of "Into the Spider-Verse" in record time 14.November 2022
MAXIM - AI tool combines various image enhancements in one model 13.November 2022
Blackmagic DaVinci Resolve 18.1 Update - with AI voice isolation 11.November 2022
Photostock portal Shutterstock gives an answer to the big question and integrates AI images 5.November 2022
AvatarCLIP: New AI generates and animates 3D avatars by text description 28.October 2022
alle Newsmeldungen zum Thema Machine Learning
1E0UPDATE 2: Black Friday and Cyber Monday deals from Avid, Laowa, Filmconvert, Viltrox and more. 25.November 2022
Mobile editing app LumaFusion: New version 3.2 - and now also for Android 11.November 2022
Even more Black Friday deals: Blackmagic, Smallrig, Pinnacle, Anton Bauer, Glidecam, Amar 10.November 2022
Stabilized Canon full-frame RF 135mm F1.8 L IS USM lens announced for January 2023 3.November 2022
New Apple Mac Pro with M2 Extreme chip with 48 CPU and 152 GPU cores and 256 GB delayed 1.November 2022
BRAW Toolbox allows native Blackmagic RAW file import under Final Cut Pro 1.November 2022
Apple unveils updates for better performance for Final Cut Pro, Compressor, iMovie, and more 27.October 2022
alle Newsmeldungen zum Thema DV Editing
1E0Mobile editing app LumaFusion: New version 3.2 - and now also for Android 11.November 2022
Video editing by text command: After text-to-image comes text-to-video 11.September 2022
LumaFusion v3.1 update - mobile editing app now offers video scopes 20.July 2022
Is AI Revolutionizing Photography? Object-based editing with the help of DALL-E 2 30.June 2022
Events @ Filmfest München: Series editing, studio tour and more 20.June 2022
Video editing via AI: Simply replace backgrounds or make objects disappear 21.January 2022
Audio Design Desk 1.9: Real-time editing tool for sound effects now with new features and Lite versi 15.January 2022
alle Newsmeldungen zum Thema Montage


[nach oben]
















Archiv Newsmeldungen

2022

December - November - October - September - August - July - June - May - April - March - February - January

2021
December - November - October - September - August - July - June - May - April - March - February - January

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

2001

2000






































deutsche Version dieser Seite: Perfekt für Musikvideos: Räumliches, objektbasiertes Videoediting per KI und 16 Kameras



last update : 6.Dezember 2022 - 12:02 - slashCAM is a project by channelunit GmbH- mail : slashcam@--antispam:7465--slashcam.de - deutsche Version