YouTube's controversial auto-dubbing feature will use AI for lip syncing

One of the most annoying features that Google rolled out in recent time on YouTube is auto-dubbing and auto-translations of content. A video that is uploaded in English by its producer may be translated automatically by YouTube so that its title and its audio are provided in a different language.
Google says that this is extending the reach of the video, which means more clicks and eyes on ads. Critics argue that they want to view the original video, not an AI generated version that offers subpar quality more often than not. Especially multi-lingua users are annoyed by the feature, as YouTube does not provide options to block the feature for languages. Extensions like YouTube Anti-Translate deal with that.
Google announced an addition to the feature today on its Made on YouTube event. Called auto-dubbing with lip sync, it aims to make auto translated videos more believable by matching the translation to the lip movement of the speaker.
Currently, YouTube's auto-translate and auto-dub feature does not take this into account. Means, the movement of the lips of a speaker do not match the spoken works and sounds that come out of the mouth.
The feature is rolling out in the coming months. It supports 20 different languages, including English, German, French, and Spanish initially, with more languages coming in the future.
Google writes: "We're making auto dubbing more realistic than ever. With upcoming lip sync technology, translated videos will now visually match the speaker's lips with the newly dubbed language, making your content more accessible and engaging for global audiences. We support dubbing across 20 different languages, and we’ll begin testing this new lip sync feature in the coming months."
A short video demonstrates the new functionality on YouTube:
Closing Words
Automatically dubbed videos will look more realistically once the feature becomes available. It seems that creators may need to enable the feature for now for their videos. Google could use it automatically in the future to make its auto-translated videos more attractive and less disliked.
Now You: What is your take on auto-translated and auto-dubbed videos? Good move to expand the reach of videos, or annoying?
What makes me furious is that YouTube doesn’t let the user set its preferences. I think only the creator can configure the default values.
All my OS (Windows, iOS and Android) are in French because I’m a native French speaker. But I listen to content in both French and English. At the present moment, YouTube is translating all the titles and descriptions without my consent and without giving me the possibility to change it. It makes me sick.
I contacted YT a few months ago. After numerous attempts to bypass bots, what I think is a human tried to make me believe he/she was not understanding my point.
If YouTube doesn’t respect content creators, imagine how they don’t give a damn about the content viewers.
I am not sure if this works with titles and descriptions, but somebody made an extension to block new auto translate feature: YouTube Anti Translate for Chromium browsers. Look it up in Chrome web store. I can not test it out because I do not see auto translate on my end yet.
I think if you use 3rd party programs like NewPipe, Grayjay, FreeTube, this feature is not on by default or maybe not even available.
I still hate the idea that a website I use is so disgustingly bad that I have to use a 3rd party standalone program to circumvent its bad experience.
I cannot accept the very idea of auto-translated and auto-dubbed videos, at least when set as default.
I could find it not only acceptable but worthy when it is an option in the video’s page or settings’ pop-up.
Video titles and descriptions should as well remain those included by the author.
IMO it is not acceptable in cultural terms nor is it legitimate regarding the respect due to the authors. Sorry but hearing a Frenchman speak American without the slightest accent is just impossible, even with lip sync.
From there on the debate is only that of personal preferences, like in the movies when some prefer sub-titles and others post-sync. Arguments exist on both sides.
Hence, Video titles, descriptions and video itself: the original ones with a button to choose another language for each. That’s not hard to implement.
Lip syncing is of course necessary once a video dubbed.
Extensions exist, at least for Firefox, to disable this incongruity:
YouTube No Translation: [https://addons.mozilla.org/en-US/firefox/addon/youtube-no-translation/]
YouTube Default Settings : [https://addons.mozilla.org/en-US/firefox/addon/youtube-default-settings/]
Neither will work with embedded YouTube videos unless I presume you allow 3rd-part cookies. I presume because I’ve never and will never allow 6rd-party cookies.
Regarding embedded YouTube videos (as well all YouTube videos) a script was available, on GitHub as well as on Greasyfork named ‘deTube Disable AI Audio’. Both have disappeared from the radar, suddenly. Fortunately those of us who had a copy can still use it flawlessly.
Agree. No Auto-translate by YouTube default. Just add a language switch button to all the setting on the bottom of videos.