Logo Logo
directory schraeg
Camcorders· Cinema-Kamera· Computers· Drohnen· GPU· Kamera-Zubehör· Video-DSLR· accessories
Compositing· Color correction· DV Editing

Shooting· Sound
Forschung· Reviews· Streaming
/// News
AI allows editing and retiming of moving persons in videos

AI allows editing and retiming of moving persons in videos

[10:57 thu,3.December 2020   by ]    

Researchers from Google and the University of Oxford have presented an interesting new algorithm for manipulating moving people in videos. The algorithm, which is based on neural networks, recognizes different moving objects in a video and assigns them to different layers - this makes a variety of interesting effects possible by manipulating the individual layers in relation to each other.


For example, different objects (including motion-correlating image objects such as their shadows and reflections!) can be removed without trace, duplicated, frozen or retimed and synchronized with each other. In the example video, some chaotically jumping persons on different trampolines become a group jumping to the beat.

The neural network performs a multitude of complex classical compositing actions: it reconstructs, for example, a hidden person walking past in the background; it assigns to each person the shadows and reflections belonging to him or her and generates a mask from this, and it even separates two temporarily overlapping shadows into individual representations.

The method allows the very easy creation of moving masks / layers of persons including their correlating tracks in the image and their simple manipulation (like retiming) in a compositing program. The possibilities of intelligent, object-based video editing and compositing using such an algorithm are enormous: for example, the movements of people in a video can be easily and completely changed afterwards - for the purpose of better image composition. And the whole thing also works with a moving camera.

Layer of people generated from videos

The downside is still the very low video resolution of 448 × 256 pixels - in order to be used in a professional environment, the algorithm would have to be further optimized or run on stronger hardware, but the research project is always a feasibility study and it is only a matter of time until an AI algorithm that is interesting for practical applications is developed to general usability.


In the video of the project it is nicely explained how the neural network is trained to recognize moving objects including correlated effects and to create its own layer out of them - this also nicely illustrates the completely different approach of Deep Learning in contrast to traditional programming.

Here the presentation of the method by Two Minute Papers:

The paper Layered Neural Rendering for Retiming People in Video will be officially presented at SIGGRAPH Asia starting on December 4th, then the program code on GitHub should also go online, which should create manipulable layers from videos in classic compositing programs.

Compositing using the generated layers

Link more infos at bei retiming.github.io

deutsche Version dieser Seite: KI ermöglicht automatisches Layering von bewegten Video-Objekten


  Vorige News lesen Nächste News lesen 
bildSigma: Drei neue 24, 35 und 65mm Vollformat-Objektive der I-Serie für DSLMs bildPhilips 326P1H und Philips 329P1H: Neue 31.5" Monitore mit integrierter USB-C Dockingstation

related news:1E0Change age, gender or facial expression of a face via AI Photo Editing 4.April 2021
Nikon: New, free image and video editing software NX-Studio incl. metadata support 4.March 2021
Video editing under Linux - Flowblade 2.8 is finished 16.February 2021
iOS video editing app LumaFusion: Version 2.4 brings HDR support and 10-bit processing 22.November 2020
Blackmagic DaVinci Resolve 17 - new color tools, faster editing, and a massive audio update 9.November 2020
Apple: iPad Air with A14 Bionic CPU: Even more performance for real-time 10 Bit H.265 4:2:2 editing? 17.September 2020
Magix Video deluxe 2021 supports 8K editing 24.August 2020
alle Newsmeldungen zum Thema Montage
1E0Frame.io launches cloud-based real-time workflow 6.April 2021
Change age, gender or facial expression of a face via AI Photo Editing 4.April 2021
Avid Media Composer 2021.3 offers new IMF workflow 29.March 2021
Three new,full-frame E-mount "prime" G lenses from Sony at 50mm, 40mm and 24mm 23.March 2021
Adobe After Effects 18.0 update brings real-time 3D design preview and multi-frame rendering 12.March 2021
Adobe Premiere Pro 15.0 brings new subtitle workflow and more 11.March 2021
Calculate audio plugins on GPU 8.March 2021
alle Newsmeldungen zum Thema DV Editing
1E0Change age, gender or facial expression of a face via AI Photo Editing 4.April 2021
AI generates photorealistic images according to description 28.March 2021
Flim - the AI search engine for film stills 28.February 2021
Topaz Labs Video Enhance AI 2.0: Video enhancements via AI at a special price 28.February 2021
Colourlab Ai 1.1 brings AI color correction via Look Designer for DaVinci Resolve 27.February 2021
JPEG starts standardisation of AI image compression - JPEG AI 11.January 2021
AI invents images according to text description - DALL-E 6.January 2021
alle Newsmeldungen zum Thema Machine Learning

[nach oben]

Archiv Newsmeldungen


April - March - February - January

December - November - October - September - August - July - June - May - April - March - February - January





















deutsche Version dieser Seite: KI ermöglicht automatisches Layering von bewegten Video-Objekten

last update : 10.April 2021 - 18:02 - slashCAM is a project by channelunit GmbH- mail : slashcam@--antispam:7465--slashcam.de - deutsche Version