Logo Logo
directory schraeg
Camcorders· Cinema-Kamera· Computers· Drohnen· GPU· Kamera-Zubehör· Video-DSLR· accessories
Compositing· Color correction· DV Editing

Shooting· Sound
Forschung· Reviews· Streaming
/// News
AI allows editing and retiming of moving persons in videos

AI allows editing and retiming of moving persons in videos

[10:57 thu,3.December 2020   by ]    

Researchers from Google and the University of Oxford have presented an interesting new algorithm for manipulating moving people in videos. The algorithm, which is based on neural networks, recognizes different moving objects in a video and assigns them to different layers - this makes a variety of interesting effects possible by manipulating the individual layers in relation to each other.


For example, different objects (including motion-correlating image objects such as their shadows and reflections!) can be removed without trace, duplicated, frozen or retimed and synchronized with each other. In the example video, some chaotically jumping persons on different trampolines become a group jumping to the beat.

The neural network performs a multitude of complex classical compositing actions: it reconstructs, for example, a hidden person walking past in the background; it assigns to each person the shadows and reflections belonging to him or her and generates a mask from this, and it even separates two temporarily overlapping shadows into individual representations.

The method allows the very easy creation of moving masks / layers of persons including their correlating tracks in the image and their simple manipulation (like retiming) in a compositing program. The possibilities of intelligent, object-based video editing and compositing using such an algorithm are enormous: for example, the movements of people in a video can be easily and completely changed afterwards - for the purpose of better image composition. And the whole thing also works with a moving camera.

Layer of people generated from videos

The downside is still the very low video resolution of 448 × 256 pixels - in order to be used in a professional environment, the algorithm would have to be further optimized or run on stronger hardware, but the research project is always a feasibility study and it is only a matter of time until an AI algorithm that is interesting for practical applications is developed to general usability.


In the video of the project it is nicely explained how the neural network is trained to recognize moving objects including correlated effects and to create its own layer out of them - this also nicely illustrates the completely different approach of Deep Learning in contrast to traditional programming.

Here the presentation of the method by Two Minute Papers:

The paper Layered Neural Rendering for Retiming People in Video will be officially presented at SIGGRAPH Asia starting on December 4th, then the program code on GitHub should also go online, which should create manipulable layers from videos in classic compositing programs.

Compositing using the generated layers

Link more infos at bei retiming.github.io

deutsche Version dieser Seite: KI ermöglicht automatisches Layering von bewegten Video-Objekten


  Vorige News lesen Nächste News lesen 
bildSigma: Drei neue 24, 35 und 65mm Vollformat-Objektive der I-Serie für DSLMs bildPhilips 326P1H und Philips 329P1H: Neue 31.5" Monitore mit integrierter USB-C Dockingstation

related news:1E0Adobe Premiere Pro Update 14.8 brings timecode support for non-standard frame rates 19.January 2021
Major update for DJI Mini 2 brings zoom for photos, more frame rates for 2.7K video and more 5.January 2021
Meike announces 35mm T2.1 -- second cine-prime for full frame 4.January 2021
Premiere Pro/Rush and Audition: betas for Apple M1 accelerate workflows 22.December 2020
Olive 0.2 Video Editor - complete rewrite with interesting features 21.December 2020
Apple: Update for Final Cut Pro and iMovie brings improved sharing to YouTube and Facebook 16.December 2020
RIFE - AI frame interpolation now 10x faster and to try out for yourself 13.December 2020
alle Newsmeldungen zum Thema DV Editing
1E0iOS video editing app LumaFusion: Version 2.4 brings HDR support and 10-bit processing 22.November 2020
Blackmagic DaVinci Resolve 17 - new color tools, faster editing, and a massive audio update 9.November 2020
Apple: iPad Air with A14 Bionic CPU: Even more performance for real-time 10 Bit H.265 4:2:2 editing? 17.September 2020
Magix Video deluxe 2021 supports 8K editing 24.August 2020
Revolutionary real-time editing tool Audio Design Desk for sound effects with free version 22.May 2020
Collaborative sound editing over the net via Soundwhale 13.May 2020
Montage: Thelma Schoonmaker about six scenes from The Irishman 14.March 2020
alle Newsmeldungen zum Thema Montage

[nach oben]

Archiv Newsmeldungen



December - November - October - September - August - July - June - May - April - March - February - January





















deutsche Version dieser Seite: KI ermöglicht automatisches Layering von bewegten Video-Objekten

last update : 20.Januar 2021 - 18:02 - slashCAM is a project by channelunit GmbH- mail : slashcam@--antispam:7465--slashcam.de - deutsche Version