Google rolls out Live Caption feature in Chrome for Windows
Google has started the rollout of its Live Caption feature in the Windows desktop version of the company's Chrome web browser. Designed to generate real-time captions automatically when enabled, Chrome's Live Caption feature displays captions for audio and video content on most sites automatically.
Chrome displays a Live Caption overlay on the screen when audio or video content plays in the browser. One caveat: the feature works only with English language content and only in the Windows version of Chrome at the time of writing.
Google revealed plans to bring it to Mac and Linux desktop versions of Chrome in the near future. No word yet on expanding language support.
Live Caption is disabled by default. Chrome desktop users can enable it in the following way:
- Load chrome://settings/accessibility in the web browser's address bar.
- Locate the "Live Caption" entry under Accessibility and toggle it with a click. A blue slider color means that the feature is enabled, white/gray that it is disabled.
- Chrome will download language components when the feature gets enabled for the first time; this may take a moment but is required for the functionality to work.
Once that is out of the way, head over to any audio or video site to get started. The Live Caption overlay is displayed automatically on any site with audio or video streams, provided that the used language is English.
You can move the overlay around using drag and drop, and expand the interface to display more text at the same time. There is also a close icon in the overlay to hide it on the screen, and Google baked the Live Caption toggle into Chrome's media playback controls at the top of the screen.
The feature appears to be a bit buggy at the time of writing. When I closed the overlay using the x-icon, it would not be displayed anymore on page reloads or even after a restart of the browser. Turned out that I had muted the player on a YouTube page, and that this seemed to prevent Live Caption functionality. Turning on audio for a brief moment returned the overlay, however.
Live Caption is displayed automatically on the screen, and the feature works even when audio is muted (apart from the bug that I discovered).
Google notes that the captions are created on the device, and that the feature works offline as well once the language components have been downloaded to the browser.
These captions in Chrome are created on-device, which allows the captions to appear as the content plays without ever having to leave your computer. Live Caption also works offline, so you can even caption audio and video files saved on your hard drive when you play them in Chrome.
Live Caption works best on sites that don't support captions by default. It is ideal for that purpose, but also great for other purposes, e.g. when you want to read but cannot listen to audio, want to read next to listening to the audio, or for accessibility purposes.
Main caveat is that it works on Windows and with English language audio only at the time of writing.
An option to disable the feature permanently for select sites, or enable it for select sites only, would be welcome. Currently, your only option is to toggle Live Captions completely or close the overlay whenever it appears.
This is so cool, and useful. As a non native EN speaker, I struggle sometimes to understand what is being said on podcasts and youtube videos (those without subtitles option).
Given that YouTube’s auto-generated captions are hilarious nonsense, I don’t expect this to be any better. If anything, probably much worse if it’s really true that they’ll be generated locally with no data sent to Google (doubtful).
It is nice indeed. Now, what would be nice would be to be able to use it independently from Chrome if, as they pretend, everything stays offline (and block it with a firewall).
You could also imagine transcribe subtitles to a text file. There are many potential use cases for this Live Caption feature
Works on Chrome Dev. Should also be pointed that this is part of Android 11 (or maybe even 10) and works everywhere, on all devices. It’s probably sending everything to Google, but is handy anyway. I’m quite impressed by its accuracy.
Speaking of. Google Search now has dark theme! Working on both Chrom* and FireFox.
This is great! Now even deaf people can be annoyed by that random ad auto playing on a website!