Logo Logo
deutsch
directory schraeg
Knowledge
Codecs
Hardware
Camcorders· Cinema-Kamera· Computers· Drohnen· GPU· Kamera-Zubehör· Video-DSLR· accessories
Software
Compositing· Color correction· DV Editing
DV-Movies

HowTo
Shooting· Sound
Misc
Forschung· KI· Reviews· Streaming
/// News
The total remix: New AI Tune-A-Video makes new videos from old clips

The total remix: New AI Tune-A-Video makes new videos from old clips

[13:07 Tue,24.January 2023   by ]    

Google and Meta had already presented their text-to-video AIs a few months ago with Imagen and Make-a-Video - these work very similarly to the text-to-image AIs such as Stable Diffsion, Midjourney or DALL-E2, which are currently experiencing a boom: images or videos are generated via text description.

tuneavideo-examples




Researchers from Singapore have now presented an interesting alternative to this type of video generation: the new algorithm, called Tune-A-Video, combines a sample video with a description of the desired result - similar to the Image-2-Image function of Stable Diffusion, in which the appearance (such as image composition and shapes of the objects) of the image to be generated by the AI is roughly specified by means of an input image and defined in more detail by text. In the case of Tune-A-Video, an own video is entered analogously - by means of a description, the individual objects in the foreground as well as the background can then be exchanged as desired.




The possibilities range from object-based video editing, i.e. targeted objects can be manipulated as in the following example of a cat wearing a hat: the cat can be doubled, the hat exchanged for another, or the facial expression changed, or the hat can be removed from the video without any gaps. Likewise, entire objects from the video can be exchanged for others, the background can be replaced by another or the entire video can be reproduced in a completely different style (for example, as a comic, oil painting, anime or pencil drawing) - in each case including all movements.

tuneavideo-cat
Cat with hat in the original and with variations


Compared to the pure video (or image) generation via text prompt, this has the advantage that the type of movement (how fast, in which movement style, from where to where) and the general image composition (where should each object be located in the image, what is the camera angle) can be specified, which are otherwise difficult to describe in this exactness via prompt. Theoretically, one&s own video as well as clips from any film can be used as a model video, because the final result can look sufficiently different to avoid being recognised as a copyright infringement or being legally *seen*. The Tune-A-Video algorithm of this kind combines the possibilities of techniques such as the animation of objects by means of motion capturing with the design and dynamic rendering of these objects.

tuneavideo-examples2


At the moment, it already delivers very good continuity of the movements and the objects, even if the replaced objects themselves are often still rendered quite incorrectly and the frame rate is very low. But as we know, with the current development speed of AIs, the quality of the results is increasing quite rapidly and should soon be sufficiently good.

Link more infos at bei tuneavideo.github.io

deutsche Version dieser Seite: Der totale Remix: Neue KI Tune-A-Video macht neue Videos aus alten Clips

  

  Vorige News lesen Nächste News lesen 
bildGroßer Vergleichstest: Upscaling per KI - welches Tool ist am besten (und kostenlos)? bildVEGAS 20 Update 2 bringt zahlreiche Neuerungen plus Rabattaktion


related news:1E0Video clip: The new Atomos monitor/recorder line-up explained: new features, applications and more, 22.September 2023
Video clip: Canon RF Cine fixed focal length explained 20.September 2023
Video clip - Adobe Premiere Pro explained: Automatically remove pauses and filler words with AI, etc. 19.September 2023
Video Clip: SmallRigs new VB99 Pro Battery, ultra-compact 60W LED and Smartphone Cage 18.September 2023
Interview: Blackmagic iPhone Camera App and Cloud explained 16.September 2023
Ghostbusters test project: real-time shooting with virtual camera in Unreal Engine 21.August 2023
Moment T-Series, new clip-on lenses for smartphones with larger sensors 17.August 2023
alle Newsmeldungen zum Thema Shooting
1E0YouTube: new editing app available - and video AI tools coming soon 22.September 2023
DALL-E 3 is here and generates more accurate AI images thanks to ChatGPT integration 21.September 2023
Video clip - Adobe Premiere Pro explained: Automatically remove pauses and filler words with AI, etc. 19.September 2023
Adobe introduces credit-based model for generative AI (and Creative Cloud gets more expensive) 18.September 2023
From text to audio: Stable Audio generates music and sound effects with AI 18.September 2023
Grass Valley announces EDIUS 11 with KI-based functions, dedicated audio editor and more 15.September 2023
Premiere Pro, After Effects and Frame.io - More AI, 3D Workspace and AWS Cloud 13.September 2023
alle Newsmeldungen zum Thema KI


[nach oben]
















Archiv Newsmeldungen

2023

September - August - July - June - May - April - March - February - January

2022
December - November - October - September - August - July - June - May - April - March - February - January

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

2001

2000






































deutsche Version dieser Seite: Der totale Remix: Neue KI Tune-A-Video macht neue Videos aus alten Clips



last update : 22.September 2023 - 18:02 - slashCAM is a project by channelunit GmbH- mail : slashcam@--antispam:7465--slashcam.de - deutsche Version