Meet Meta SeamlessM4T: Multilangual translator
Meta has just dropped an impressive new tool called SeamlessM4T. This snazzy AI model can transcribe and translate nearly 100 languages, and it's a big step forward in making language barriers a thing of the past.
The company claims that SeamlessM4T, Massively Multilingual and Multimodal Machine Translation, can convert voice to text and text to speech for almost 100 languages. It identifies 100 input languages and turns them into 35 output languages for speech-to-speech and text-to-speech functions.
Because it is distributed under the Creative Commons CC BY-NC 4.0 license, researchers are free to improve it. The information for SeamlessAlign, part of Meta's open translation dataset, was also made available together with SeamlessM4T. As opposed to existing big translation models that divide translation among several systems, SeamlessM4T completes the full translation work in a single operation, according to Meta, making it "a significant breakthrough."
Meta SeamlessM4T works pretty fast
If SeamlessM4T works as intended, one of its intriguing aspects is its supposed capacity to detect code-switching or the transition between two or more languages inside a single phrase. For instance, Meta showed in a video how the model can immediately distinguish between Hindi, Telugu, and English.
It boasts the potential to instantly discern when a speaker traverses languages, as demonstrated in a captivating video where it effortlessly distinguished between Hindi, Telugu, and English. This capability could revolutionize cross-cultural communication, fostering a world where language is no longer a barrier.
"In keeping with our approach to open science, we’re publicly releasing SeamlessM4T under a research license to allow researchers and developers to build on this work. We’re also releasing the metadata of SeamlessAlign, the biggest open multimodal translation dataset to date, totaling 270,000 hours of mined speech and text alignments," says Meta.
What is Llama2: Meta's AI explained
To produce more lightweight models and manage more information, Meta claimed it modified its Fairseq sequence modeling toolbox to generate SeamlessM4T.
According to Meta, it created a system that can recognize harmful or delicate phrases when creating SeamlessM4T. According to the definition given by Meta, toxic words are those whose "translation may incite hate, violence, profanity, or abuse." The objective is to be able to identify instances where toxicity introduced by the output translation has not previously existed in the source content.
Advertisement
