VLC to add support for offline AI subtitles and translations
VideoLAN's open-source video player has reached 6 billion downloads. The developers celebrated the milestone by announcing a new feature at CES 2025, AI generated subtitles.
Subtitles are useful when you want to watch a movie or a TV show that is in a foreign language, or if you have troubles hearing. But subtitles may not always be available for the video that you wanted to watch, and sometimes they can be hardcoded or use weird fonts or colors. Additionally, a good chunk of Internet users may not know how to find, download, and integrate subtitles for local media.
Live Captions
Windows 10 and Windows 11 have a feature called live captions, you can access it with the following hotkey: Windows + Ctrl + L. It is a handy feature that you can use for any media, including streaming videos on YouTube, Netflix, and other video streaming sites. Admittedly, it sometimes struggles to provide accurate subtitles, but it is better than nothing and improving. Android, iOS, macOS also have similar live captioning features.
VLC automatic subtitle generation and translations
Maybe VLC's offline subtitle generator can be a better solution for this. But it is not the first to introduce such a feature. There are other tools that use Open AI's Whisper to generate subtitles, like Vibe or Subtitler. But these apps are not user-friendly per se. What VLC seems to have built could be an easier solution.
VLC to get automatic AI subtitling and real-time translations
VLC's GitLab has some merge requests related to implementing a speech-to-text technology in the video player. It will use Whisper, but it's not relying on OpenAI's Whisper model. Instead, VLC will be using whisper.cpp, a C-based fork of Open AI's Whisper automatic speech recognition (ASR) technology.
VideoLAN's AI subtitling tool will work offline without connecting to online services, i.e. without any requirements for an account or service. That also means it will be privacy-friendly. Besides, it also eliminates the need to manually search for subtitles, download them, and enable them for the video.
It's not just about generating subtitles, what's more impressive is that VLC's AI capabilities will also translate the subtitles in real-time. The feature will support over 100 languages, this could allow users to understand dialogs in any language. Imagine you are travelling or don't have access to the internet, and VLC can not only provide the subtitles for videos, but also translate them offline. That is remarkable. The requirements for the feature have not been revealed, but it will likely require an optional download that contains the AI model required to generate the subtitles, and translate them.
AI subtitling in VLC is not available just yet, it will be added in the future. You can watch the announcement video on VideoLAN's X account and LinkedIn page.
Did you know that the latest version of PotPlayer also supports automatic subtitling? It also uses the Whisper models.
I called Amazon Prime Video AI recap a cool use of artificial intelligence, and I think VLC's AI subtitles is even better. This is another example of how AI can be beneficial when it does not invade your privacy. But the success of the feature would depend on the accuracy of the generated subtitles. Hopefully it will work as promised.
More of this AI bullshit, I’m sure the actual quality of these “subtitles” and “translations” is laughably bad (like those from OpenSubtitles)
Guess you’ve never seen what butchering that localizers do to foreign media while injecting their views over accuracy.
As long as there are no connections, logs, or any reporting or conectivity of any kind at any time, using tjis or any other software or application. This could be a good thing. But I am concerned as the value of any data from any use of what one does on a computer is very valuable, and the temptation for companies to obtain this data is very high. So just be aware and diligent in continuing to veriy that you data is not being taken.
I use SMPlayer, but on Linux as it’s a superior OS. When I used Windows, I still used SMPlayer, it’s better than VLC. Only on android I use VLC, because there is no Employer for Android.
Has VLC implemented an official dark theme on Windows yet? A year ago it was only available on MacOS. A deal-breaker for me. Using MPC-BE at the moment.
MPC-HC (fork by clsid2) has a dark mode also and some different features to MPC-BE if you are interested then try a portable release.
They are both excellent players.
https://www.nikse.dk/subtitleedit
subtitleedit uses whisper/faster-whisperer and other models and is free. fast way to generate SRT files.
SMPlayer is still better, sorry.
VLC is by far the best player ever, nothing other can be compared in equal conditions.