Logo Logo
directory schraeg
Camcorders· Cinema-Kamera· Computers· Drohnen· GPU· Kamera-Zubehör· Video-DSLR· accessories
Compositing· Color correction· DV Editing

Shooting· Sound
Forschung· Reviews· Streaming
/// News
VFX made easy: animate faces and fake voices with AI

VFX made easy: animate faces and fake voices with AI

[11:28 Thu,22.September 2022   by ]    

Alternative titles:
Quick and easy: Special Effects by AI
Quick and easy: VFX by AI
AI instead of motion capture

We recently presented a short clip of a virtual dynamic fashion show, which shows how easily and yet effectively different AI tools can be used in combination to create interesting effects in videos.


Filmmaker and musician Scott Lighthiser has combined three other free AI apps for for simply made but quite spectacular special effects in the following clips here: the AI image generator Stable Diffusion, the style transfer tool Ebysnth and the AI voice transformer Koe Recast.

Each was based on a short video of himself scaled to 1,024 x 1,024. With the help of Stable Diffusion per Google Colab and its img2img function, which makes it possible to change an image specifically per AI, Lighthiser then created completely different but precisely fitting variations of the speaker, once a female Greek marble statue, then an old man and last but not least a bloody zombie. These three images were then upscaled to 1,024 x 1,024 pixels by AI and used as style sheets in Ebsynth, which were then applied to the entire video. Finally, to change his voice to match, Lighthiser used the AI-powered app Koe Recast, which allows you to change its sound in real time as desired.


Even more blatant is his following horror animation, which transforms not only his face but his entire body into that of an undead (the prompt for this, by the way, was "The dry creature, film still from the movie directed by Denis Villeneuve and David Cronenberg with art direction by H. R. Giger and Bernie Wrightson"):

The horror motif is particularly suitable for these demos because dynamic artifacts created by an imperfect style transfer via Ebsynth, which would otherwise be distracting, tend to enhance the shock effect here.

The classic workflow would have been a lot more complicated: first the speaker&s face would have to be read in as a 3D model via face capturing, then a 3D model of the new face would have to be created using a 3D modeling program, which would then be mapped onto the original face and rendered. All of this requires a lot of specialized knowledge in addition to the right tools.


In contrast, using the simple AI workflow and relatively easy-to-use tools, very different alienated versions of the original video were created in a short time - a small foretaste of the extensive possibilities of the currently explosively developing AI tools in the field of image - and also video.

The whole thing is just a small demonstration of what effects can be created by combining multiple AI tools - before there are even any AIs specialized entirely in video that can freely create or edit entire videos on demand using only text description - much like Stable Diffusion or DALL-E2 can do with images right now. If such tools come in the future and the resolution and image quality are improved even further, then after the revolution in filmmaking, which has so far been driven primarily by hardware, the next stage will be ignited with the help of AI tools by making professional and yet easy-to-create special effects (and also animations) accessible to a broad mass.

Link more infos at bei twitter.com

deutsche Version dieser Seite: VFX leicht gemacht: mit KI Gesichter animieren und Stimmen synthetisieren


  Vorige News lesen Nächste News lesen 
bildNeue Nvidia RTX 4090: Doppelt so schnell wie RTX 3090 Ti - aber 1.949 Euro teuer bildDJI Osmo Mobile 6: Neuer Smartphone-Gimbal mit verbesserter Objektverfolgung

related news:1E0Heavy Metal meets Midjourney: Music video made from 10,000 AI-generated images 4.December 2022
Meta Encodec uses AI to compress audio files significantly more than MP3 21.November 2022
AI copies movie style of "Into the Spider-Verse" in record time 14.November 2022
MAXIM - AI tool combines various image enhancements in one model 13.November 2022
Blackmagic DaVinci Resolve 18.1 Update - with AI voice isolation 11.November 2022
Photostock portal Shutterstock gives an answer to the big question and integrates AI images 5.November 2022
AvatarCLIP: New AI generates and animates 3D avatars by text description 28.October 2022
alle Newsmeldungen zum Thema Machine Learning
1E0Maxon One Tools updated - Trapcode Suite 17, VFX Suite 2 and more 16.September 2021
Audio Design Desk 1.7: Real-time editing tool for SFX now with DAW synchronization 18.July 2021
Update: Red Giant VFX Suite 1.5 offers new Lens Distortion tool 18.May 2020
VFX-Oscar: Which films and what they are nominated for 9.February 2020
About the award-winning editing of Parasite -- live first edting on set, hidden VFX cuts and FCP 7 26.January 2020
The first computer-generated special effects 29.December 2019
HitFilm Express 13 -- free edting- and VFX-program gets an update 15.September 2019
alle Newsmeldungen zum Thema Special effects

[nach oben]

Archiv Newsmeldungen


December - November - October - September - August - July - June - May - April - March - February - January

December - November - October - September - August - July - June - May - April - March - February - January






















deutsche Version dieser Seite: VFX leicht gemacht: mit KI Gesichter animieren und Stimmen synthetisieren

last update : 7.Dezember 2022 - 18:42 - slashCAM is a project by channelunit GmbH- mail : slashcam@--antispam:7465--slashcam.de - deutsche Version