Logo Logo
directory schraeg
Camcorders· Cinema-Kamera· Computers· Drohnen· GPU· Kamera-Zubehör· Video-DSLR· accessories
Compositing· Color correction· DV Editing

Shooting· Sound
Forschung· KI· Reviews· Streaming
/// News
VideoFusion: First open source video AI is here - and also runs on the home PC

VideoFusion: First open source video AI is here - and also runs on the home PC

[13:41 Mon,20.March 2023   by ]    

A Chinese research team has published a new text-to-video AI with 1.7 billion parameters, i.e. it can only generate video via text input. Similar algorithms have already been presented by Meta with Make-a-Video and Google with Imagen and Phenaki, but the special thing about VideoSAcope (here a demonstration on a Chinese website, translated to German by Google) is that the source code including the corresponding models is freely available for download. The new method uses for the generation of videos the well known diffusion method, which is also used by the well known AIs for image generation like for example Stable Diffusion.


Generating videos on your own PC

Those who have some experience in configuring AI algorithms including models can try VideoFusion on their own PC - however, a high-performance GPU with at least 16 GB VRAM or 8 GB with half accuracy as well as 16 GB RAM is still a prerequisite. On an Nvidia RTX 3090 graphics card, generating a xxx second clip takes about 23s. Here is a manual for the installation on your own PC. The generated video is output as .MP4 file, for playback the open source free VLC Player is recommended.

Alternatively, there is already a implementation on the well-known AI portal Hugginface, which you can use yourself for free - Allersbergs generation there takes a relatively long time due to the shared GPU power. But if you need fast video generation, you can rent an Nvidia A10G GPU with 46 GB VRAM (or a more powerful A100 GPU) for around per hour.

Monkey learning to play the piano:


The researchers themselves restrict that VideoFusion cannot produce perfect movie and TV quality, nor can it produce text in the video. Only English is supported as an input language at the moment. It is also obvious that the image quality and resolution (128x 128) is still relatively low and reminiscent of the early days of image generation by AI - but experience in the field of AI shows how fast progress is there, especially if the associated program and model - as was already the case with Image AI Stable Diffusion - is available as open source and can thus be further developed and improved by anyone.

Robot dancing in times square:
modelscope.cn/api/v1/models/damo/text-to-video-synthesis/repo?Revision=master&FilePath=./samples/Monkey_learning_to_play_the_piano._2.gif&View=true "Y

At the moment VideoFusion also uses only 1.7 billion parameters - for comparison: DALL E2, which can only generate images, was trained with more than 10 billion parameters, so there is still a lot of room for further quality improvements just by using more parameters.

Shutterstock logo in generated videos

Copyright by Shutterstock?

It is noticeable that on a large part of the demonstrated videos of VideoFusion the logo of Shutterstock, the largest stock photo portal, is too prominently visible, indicating that their images were included in the image and video training material used ( LAION5B, ImageNet and Webvid).


It will be interesting to see how Shutterstock reacts to such videos, which are potentially legally contestable - on the one hand for unauthorized use of Shutterstock images as training material and on the other hand for unauthorized and potentially business-damaging use of the logo.

Link more infos at bei paperswithcode.com

deutsche Version dieser Seite: VideoFusion: Erste Open Source Video-KI ist da - und läuft auch auf dem Heim-PC


  Vorige News lesen Nächste News lesen 
bildRuinieren schlechte Projektionen das Kino? bildPanasonic: Neuer organischer Sensor liefert verbesserte Farbwiedergabe

related news:1E0Spacetop: Notebook mit virtuellem 100 19.May 2023
Now live streaming at 4K and 60 fps with Nvidia RTX 40 graphics cards. 8.May 2023
New NVIDIA AI can simulate tennis matches - using only TV images 7.May 2023
Radeon RX 7950 XTX and XT - AMD accidentally reveals RTX 4090 competitor 3.May 2023
NVIDIA RTX 5000 ADA - graphics card with 32 GB ram in the starting blocks 24.April 2023
Soon to be cinematic? New NVIDIA AI creates high-definition videos 20.April 2023
Nvidia RTX 4070 ships - Good performance, but higher prices for the mid-range 14.April 2023
alle Newsmeldungen zum Thema Computers
1E0Making of - How the viral Wes Anderson AI videos are created 29.May 2023
Microsoft Olive: New free tool doubles performance of Stable Diffusion 24.May 2023
Generative Fill - AI image enhancement in Photoshop Beta, easy for everyone! 23.May 2023
Diffusae - Stable diffusion for After Effects plugin 23.May 2023
Synthetic AI voices are competing with professional speakers 21.May 2023
Drag your GAN - Change AI motifs simply by dragging them 19.May 2023
Stable Diffusion rejuvenates Harrison Ford 16.May 2023
alle Newsmeldungen zum Thema KI

[nach oben]

Archiv Newsmeldungen


May - April - March - February - January

December - November - October - September - August - July - June - May - April - March - February - January























deutsche Version dieser Seite: VideoFusion: Erste Open Source Video-KI ist da - und läuft auch auf dem Heim-PC

last update : 30.Mai 2023 - 12:02 - slashCAM is a project by channelunit GmbH- mail : slashcam@--antispam:7465--slashcam.de - deutsche Version