DeepFake makes you multilingual: Google Translator AI lets anyone speak in foreign languages
[16:20 Thu,11.May 2023 by Thomas Richter]
New Google Translator AI lets anyone speak in foreign languages via DeepFake
At its developer conference, Google presented, among other things, the - still experimental - "Universal Translator", a digital translator that combines several AI functions. It is able not only to translate spoken words in videos into another language, but also to put them back into the mouth of the original speaker in his own voice - but in the new language - and also to synchronize the lip movements appropriately.
In the professional film field, there are have been for some time services like Flawless AI, which provide visual dubbing for motion pictures, matching an actor&s lip movements to the text spoken by the voice actor via AI. Google&s "Universal Translator", however, adds automatic translation including cloning of the original voice including the respective tone of voice, as is also done, for example, by Eleven Labs, and is not aimed at the cinema, but at YouTube videos.
The new algorithm was therefore also demonstrated using a lecture from an online course that Google originally produced in English together with the University of Arizona. Thanks to automatic translation integrated into YouTube, for example, such courses could be offered in any language in the future and thus be available to a much wider audience.
Many AI techniques in one tool
. Google&s "Universal Translator" uses several already existing techniques to achieve its goal. First, the spoken word is transcribed into text and the speaker&s voice, including intonation, is analyzed. Then, this text is translated into the desired language and new ones are synthesized in the speaker&s voice. Finally, lip movements matching the new audio are generated and fused with the face of the person speaking.
Here is the introduction: of the Universal Translator (1:14:29):
?t=4479
The dangers
. But in addition to its many practical uses, such technology also has great potential for misuse. Google, however, is well aware of the dangers of such automated technologies, which is why, as part of its "Bold and responsible AI" dogma, only authorized partners have access to the "Universal Translator" for the time being, which is still in an experimental state anyway and is not perfect. Google wants to provide its AI-generated content with an invisible watermark that identifies it as AI-generated.
However, since all the necessary techniques are already known and either available as research work or even as an open source algorithm, experience suggests that it won&t be long before a similar tool that combines them all is released for free. Individual DeepFake videos including simulated voices are already in circulation.
;)
;)