.footer { } Logo Logo
deutsch
/// News

Generative Omnimatte: AI breaks down each video into separable layers

[11:43 Tue,26.November 2024   by Rudi Schmidts]    

We're already used to the fact that AI is now able to extract individual instrument tracks from fully mastered stereo songs quite convincingly. So we shouldn't be surprised if it is now possible to split a video into individual layers. And Generative Omnimatte promises no less than this.

gen_omnimatte_189
Generative Omnimatte: AI breaks down each video into separable layers





This novel AI model attempts to split a video clip into semantically meaningful layers that contain individual objects and the associated effects such as shadows and reflections. Existing methods previously required at least a static background or were based on a pose or depth estimation for the layers. In addition, previous methods failed due to a fundamental problem: dynamic hidden areas between objects or layers cannot be completed as they do not contain any knowledge about the hidden areas.

This is set to change with Generative Omnimatte, which can also complete missing or unknown content in moving images using generative AI. This is referred to as “supplementation through generative prior knowledge”.



Generative Omnimatte requires no information on camera position or depth and creates complete layers - including mostly convincing completions of hidden dynamic areas.

Mind you, this does not work by splitting up existing composites but with any video recording. The published paper and the special project website with demos show a wide range of arbitrarily recorded videos with soft shadows, shiny reflections, splash water and more.

Researchers from Google DeepMind, the University of Maryland College Park and the Weizmann Institute of Science are behind Generative Omnimatte. Although such publications “only” represent the current state of research, it is foreseeable that we will soon find generative de-compositing technologies as usable tools in our editing and compositing programs.



deutsche Version dieser Seite: Generative Omnimatte: KI zerlegt jedes Video in Ebenen-Objekte

  



[nach oben]












Archiv Newsmeldungen

2024

December - November - October - September - August - July - June - May - April - March - February - January

2023
December - November - October - September - August - July - June - May - April - March - February - January

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

2001

2000






































deutsche Version dieser Seite: Generative Omnimatte: KI zerlegt jedes Video in Ebenen-Objekte



last update : 9.Dezember 2024 - 08:02 - slashCAM is a project by channelunit GmbH- mail : slashcam@--antispam:7465--slashcam.de - deutsche Version