Logo Logo
directory schraeg
Camcorders· Cinema-Kamera· Computers· Drohnen· GPU· Kamera-Zubehör· Video-DSLR· accessories
Compositing· Color correction· DV Editing

Shooting· Sound
Forschung· Reviews· Streaming
/// News
VFX made easy: animate faces and fake voices with AI

VFX made easy: animate faces and fake voices with AI

[11:28 Thu,22.September 2022   by ]    

Alternative titles:
Quick and easy: Special Effects by AI
Quick and easy: VFX by AI
AI instead of motion capture

We recently presented a short clip of a virtual dynamic fashion show, which shows how easily and yet effectively different AI tools can be used in combination to create interesting effects in videos.


Filmmaker and musician Scott Lighthiser has combined three other free AI apps for for simply made but quite spectacular special effects in the following clips here: the AI image generator Stable Diffusion, the style transfer tool Ebysnth and the AI voice transformer Koe Recast.

Each was based on a short video of himself scaled to 1,024 x 1,024. With the help of Stable Diffusion per Google Colab and its img2img function, which makes it possible to change an image specifically per AI, Lighthiser then created completely different but precisely fitting variations of the speaker, once a female Greek marble statue, then an old man and last but not least a bloody zombie. These three images were then upscaled to 1,024 x 1,024 pixels by AI and used as style sheets in Ebsynth, which were then applied to the entire video. Finally, to change his voice to match, Lighthiser used the AI-powered app Koe Recast, which allows you to change its sound in real time as desired.


Even more blatant is his following horror animation, which transforms not only his face but his entire body into that of an undead (the prompt for this, by the way, was "The dry creature, film still from the movie directed by Denis Villeneuve and David Cronenberg with art direction by H. R. Giger and Bernie Wrightson"):

The horror motif is particularly suitable for these demos because dynamic artifacts created by an imperfect style transfer via Ebsynth, which would otherwise be distracting, tend to enhance the shock effect here.

The classic workflow would have been a lot more complicated: first the speaker&s face would have to be read in as a 3D model via face capturing, then a 3D model of the new face would have to be created using a 3D modeling program, which would then be mapped onto the original face and rendered. All of this requires a lot of specialized knowledge in addition to the right tools.


In contrast, using the simple AI workflow and relatively easy-to-use tools, very different alienated versions of the original video were created in a short time - a small foretaste of the extensive possibilities of the currently explosively developing AI tools in the field of image - and also video.

The whole thing is just a small demonstration of what effects can be created by combining multiple AI tools - before there are even any AIs specialized entirely in video that can freely create or edit entire videos on demand using only text description - much like Stable Diffusion or DALL-E2 can do with images right now. If such tools come in the future and the resolution and image quality are improved even further, then after the revolution in filmmaking, which has so far been driven primarily by hardware, the next stage will be ignited with the help of AI tools by making professional and yet easy-to-create special effects (and also animations) accessible to a broad mass.

Link more infos at bei twitter.com

deutsche Version dieser Seite: VFX leicht gemacht: mit KI Gesichter animieren und Stimmen synthetisieren


  Vorige News lesen Nächste News lesen 
bildNeue Nvidia RTX 4090: Doppelt so schnell wie RTX 3090 Ti - aber 1.949 Euro teuer bildDJI Osmo Mobile 6: Neuer Smartphone-Gimbal mit verbesserter Objektverfolgung

related news:1E0Deepfake: Bruce Willis digital twin appears in commercials 1.October 2022
First the image revolution, then the video revolution: New meta AI 30.September 2022
Whisper: New free AI turns speech into text and automatically translates into all languages 26.September 2022
AI images with stable diffusion can now be used directly ni Photoshop and Gimp via plugin 18.September 2022
Generate and edit personalized images via AI 15.September 2022
Free image generation via AI: Stable Diffusion now with easy Windows Installer 13.September 2022
Insta360 X3 - 360° actioncam with new image sensors and KI functions 11.September 2022
alle Newsmeldungen zum Thema Machine Learning
1E0Maxon One Tools updated - Trapcode Suite 17, VFX Suite 2 and more 16.September 2021
Audio Design Desk 1.7: Real-time editing tool for SFX now with DAW synchronization 18.July 2021
Update: Red Giant VFX Suite 1.5 offers new Lens Distortion tool 18.May 2020
VFX-Oscar: Which films and what they are nominated for 9.February 2020
About the award-winning editing of Parasite -- live first edting on set, hidden VFX cuts and FCP 7 26.January 2020
The first computer-generated special effects 29.December 2019
HitFilm Express 13 -- free edting- and VFX-program gets an update 15.September 2019
alle Newsmeldungen zum Thema Special effects

[nach oben]

Archiv Newsmeldungen


October - September - August - July - June - May - April - March - February - January

December - November - October - September - August - July - June - May - April - March - February - January






















deutsche Version dieser Seite: VFX leicht gemacht: mit KI Gesichter animieren und Stimmen synthetisieren

last update : 2.Oktober 2022 - 11:24 - slashCAM is a project by channelunit GmbH- mail : slashcam@--antispam:7465--slashcam.de - deutsche Version