Logo Logo
directory schraeg
Camcorders· Cinema-Kamera· Computers· Drohnen· GPU· Kamera-Zubehör· Video-DSLR· accessories
Compositing· Color correction· DV Editing

Shooting· Sound
Forschung· KI· Reviews· Streaming
/// News
The total remix: New AI Tune-A-Video makes new videos from old clips

The total remix: New AI Tune-A-Video makes new videos from old clips

[13:07 Tue,24.January 2023   by ]    

Google and Meta had already presented their text-to-video AIs a few months ago with Imagen and Make-a-Video - these work very similarly to the text-to-image AIs such as Stable Diffsion, Midjourney or DALL-E2, which are currently experiencing a boom: images or videos are generated via text description.


Researchers from Singapore have now presented an interesting alternative to this type of video generation: the new algorithm, called Tune-A-Video, combines a sample video with a description of the desired result - similar to the Image-2-Image function of Stable Diffusion, in which the appearance (such as image composition and shapes of the objects) of the image to be generated by the AI is roughly specified by means of an input image and defined in more detail by text. In the case of Tune-A-Video, an own video is entered analogously - by means of a description, the individual objects in the foreground as well as the background can then be exchanged as desired.

The possibilities range from object-based video editing, i.e. targeted objects can be manipulated as in the following example of a cat wearing a hat: the cat can be doubled, the hat exchanged for another, or the facial expression changed, or the hat can be removed from the video without any gaps. Likewise, entire objects from the video can be exchanged for others, the background can be replaced by another or the entire video can be reproduced in a completely different style (for example, as a comic, oil painting, anime or pencil drawing) - in each case including all movements.

Cat with hat in the original and with variations

Compared to the pure video (or image) generation via text prompt, this has the advantage that the type of movement (how fast, in which movement style, from where to where) and the general image composition (where should each object be located in the image, what is the camera angle) can be specified, which are otherwise difficult to describe in this exactness via prompt. Theoretically, one&s own video as well as clips from any film can be used as a model video, because the final result can look sufficiently different to avoid being recognised as a copyright infringement or being legally *seen*. The Tune-A-Video algorithm of this kind combines the possibilities of techniques such as the animation of objects by means of motion capturing with the design and dynamic rendering of these objects.


At the moment, it already delivers very good continuity of the movements and the objects, even if the replaced objects themselves are often still rendered quite incorrectly and the frame rate is very low. But as we know, with the current development speed of AIs, the quality of the results is increasing quite rapidly and should soon be sufficiently good.

Link more infos at bei tuneavideo.github.io

deutsche Version dieser Seite: Der totale Remix: Neue KI Tune-A-Video macht neue Videos aus alten Clips


  Vorige News lesen Nächste News lesen 
bildGroßer Vergleichstest: Upscaling per KI - welches Tool ist am besten (und kostenlos)? bildVEGAS 20 Update 2 bringt zahlreiche Neuerungen plus Rabattaktion

related news:1E0Sony Star Sphere: Take your own photos from a satellite 8.January 2023
Hollyland Lark M1: Affordable two-channel wireless clip-on microphone system 6.July 2022
NAB 2022 Interview: Camera to Cloud, Frame IO and Premiere Pro 2.May 2022
NAB 2022 Interview: Tilta's mobile, affordable shoulder rest for DSLMs to compact cine cams 30.April 2022
NAB 2022 Tutorialclip/Interview: How does Frame.IO work in Premiere Pro? 28.April 2022
NAB 2022 Interview: Atomos and Shogun Connect - powerful cloud tools coming 26.April 2022
NAB 2022 interview: Blackmagic Design - User Senarios for the New Cloud Products 25.April 2022
alle Newsmeldungen zum Thema Shooting
1E0New NVIDIA Eye Contact AI Effect: Bye bye Teleprompter? 25.January 2023
Big comparison test: upscaling via AI - which tool is best? Free or Paid? 23.January 2023
OpenAI VALL-E: New AI mimics any voice - using only 3s voice sample 9.January 2023
New NVIDIA Video Super Resolution AI scales video from 1080p to 4K 6.January 2023
AMD's mobile Ryzen 70xx with Zen 4, RDNA 3 and XDNA AI 5.January 2023
Photoshoots plus AI: Is this what the future of photography looks like? 27.December 2022
New Disney AI can age - or rejuvenate - actors in seconds 14.December 2022
alle Newsmeldungen zum Thema Machine Learning

[nach oben]

Archiv Newsmeldungen



December - November - October - September - August - July - June - May - April - March - February - January























deutsche Version dieser Seite: Der totale Remix: Neue KI Tune-A-Video macht neue Videos aus alten Clips

last update : 28.Januar 2023 - 09:45 - slashCAM is a project by channelunit GmbH- mail : slashcam@--antispam:7465--slashcam.de - deutsche Version