The new language translation tool from ElevenLabs can convert spoken content into another language in a matter of minutes, preserving the sound of the original speaker's voice. This means that anyone can now get their hands on a simple tool to "localize" their own videos internationally.
At least that's the theory. Fortunately, you can try the service yourself since recently, although the free conversion seems to be limited to about one minute of the material.
For your own tests, you can upload an audio or video file, or simply provide a link to it. Youtube, TikTok, X(Twitter) and Vimeo links can also be used as input without any detours. Optionally you can select the input language as well as one of 29 desired output languages and after a calculation pause you can listen to the audio track in another language and if necessary also see it.
Output languages
In the clips we tested, the lips sometimes seemed more asynchronous than the definitely hand-picked examples of the manufacturer - which is of course also not an easy task - the content translation must after all "fit" on the unchanged lips. In addition, the content translation quality is more reminiscent of a simultaneous interpreter than a high-quality translation ala DeepL.
Nevertheless, the result is often astonishing. For example, an American with us retained (s)an American dialect in the German pronunciation, which made the clip much more authentic than a typical, clean German AI pronunciation. And even despite its obvious weaknesses, there are probably already a few suitable use cases for the new tool. For example, where the linguistic content is not particularly complex and the sentences are relatively short....
In this example, you can see the AI dubbing in practical use and can switch between the generated languages by selecting the audio track in Youtube...
;)
;)