Logo Logo
directory schraeg
Camcorders· Cinema-Kamera· Computers· Drohnen· GPU· Kamera-Zubehör· Video-DSLR· accessories
Compositing· Color correction· DV Editing

Shooting· Sound
Forschung· Reviews· Streaming
/// News
AI allows editing and retiming of moving persons in videos

AI allows editing and retiming of moving persons in videos

[10:57 thu,3.December 2020   by ]    

Researchers from Google and the University of Oxford have presented an interesting new algorithm for manipulating moving people in videos. The algorithm, which is based on neural networks, recognizes different moving objects in a video and assigns them to different layers - this makes a variety of interesting effects possible by manipulating the individual layers in relation to each other.


For example, different objects (including motion-correlating image objects such as their shadows and reflections!) can be removed without trace, duplicated, frozen or retimed and synchronized with each other. In the example video, some chaotically jumping persons on different trampolines become a group jumping to the beat.

The neural network performs a multitude of complex classical compositing actions: it reconstructs, for example, a hidden person walking past in the background; it assigns to each person the shadows and reflections belonging to him or her and generates a mask from this, and it even separates two temporarily overlapping shadows into individual representations.

The method allows the very easy creation of moving masks / layers of persons including their correlating tracks in the image and their simple manipulation (like retiming) in a compositing program. The possibilities of intelligent, object-based video editing and compositing using such an algorithm are enormous: for example, the movements of people in a video can be easily and completely changed afterwards - for the purpose of better image composition. And the whole thing also works with a moving camera.

Layer of people generated from videos

The downside is still the very low video resolution of 448 × 256 pixels - in order to be used in a professional environment, the algorithm would have to be further optimized or run on stronger hardware, but the research project is always a feasibility study and it is only a matter of time until an AI algorithm that is interesting for practical applications is developed to general usability.


In the video of the project it is nicely explained how the neural network is trained to recognize moving objects including correlated effects and to create its own layer out of them - this also nicely illustrates the completely different approach of Deep Learning in contrast to traditional programming.

Here the presentation of the method by Two Minute Papers:

The paper Layered Neural Rendering for Retiming People in Video will be officially presented at SIGGRAPH Asia starting on December 4th, then the program code on GitHub should also go online, which should create manipulable layers from videos in classic compositing programs.

Compositing using the generated layers

Link more infos at bei retiming.github.io

deutsche Version dieser Seite: KI ermöglicht automatisches Layering von bewegten Video-Objekten


  Vorige News lesen Nächste News lesen 
bildSigma: Drei neue 24, 35 und 65mm Vollformat-Objektive der I-Serie für DSLMs bildPhilips 326P1H und Philips 329P1H: Neue 31.5" Monitore mit integrierter USB-C Dockingstation

related news:1E0Postproduktion in der Cloud mit Adobe Premiere Pro per Qumulo Studio Q 5.June 2021
Change age, gender or facial expression of a face via AI Photo Editing 4.April 2021
Nikon: New, free image and video editing software NX-Studio incl. metadata support 4.March 2021
Video editing under Linux - Flowblade 2.8 is finished 16.February 2021
iOS video editing app LumaFusion: Version 2.4 brings HDR support and 10-bit processing 22.November 2020
Blackmagic DaVinci Resolve 17 - new color tools, faster editing, and a massive audio update 9.November 2020
Apple: iPad Air with A14 Bionic CPU: Even more performance for real-time 10 Bit H.265 4:2:2 editing? 17.September 2020
alle Newsmeldungen zum Thema Montage
1E0Blackmagic RAW 2.1 available - with improvements for Premiere and Apple's M1 24.June 2021
Adobe introduces new UI design for Premiere Pro 22.June 2021
Magix Video X Pro 13 brings improved GPU engine and new panorama storyboard 22.June 2021
Final Cut Pro update 10.5.3 brings among other things own editor for metadata columns 19.June 2021
Razer Blade 14 Laptop mit Ryzen 9 CPU, Nvidia RTX 3080 GPU und 100% DCI-P3 Display 18.June 2021
The Colorist Guide to DaVinci Resolve 17 - Good grading knowledge for free 17.June 2021
Adobe: New free tutorial series for Premiere Pro, After Effects, and Rush 14.June 2021
alle Newsmeldungen zum Thema DV Editing
1E0In color and 4K: Pornhub restores over 100 years old porn movies via AI 11.June 2021
Flim: AI-based film still search gets intelligence upgrade 4.June 2021
Professional AI speakers for everyone? MARVEL.ai 27.May 2021
Visual dubbing by AI makes every actor a native speaker 25.May 2021
Colourlab Ai 1.2: New camera profile tool and new formats 30.April 2021
Dynascore Musik-Engine -- AI times background music to videos 19.April 2021
Simon Says : Audio transcription via AI now available in Adobe Premiere Pro 18.April 2021
alle Newsmeldungen zum Thema Machine Learning

[nach oben]

Archiv Newsmeldungen


June - May - April - March - February - January

December - November - October - September - August - July - June - May - April - March - February - January





















deutsche Version dieser Seite: KI ermöglicht automatisches Layering von bewegten Video-Objekten

last update : 24.Juni 2021 - 12:02 - slashCAM is a project by channelunit GmbH- mail : slashcam@--antispam:7465--slashcam.de - deutsche Version