What is Meta's MusicGen and how to use it?

Get ready to enter a new era in the music industry as Meta has recently announced the launch of MusicGen, an AI-powered music generator that promises to revolutionize the way we create and consume music.

This announcement comes on the heels of intense competition in the AI music generation space, and Meta's offering stands out as an impressive contender.

Here is how Felix Kreuk explains Meta MusicGen on Twitter:

We present MusicGen: A simple and controllable music generation model. MusicGen can be prompted by both text and melody.
We release code (MIT) and models (CC-BY NC) for open research, reproducibility, and for the music community: https://t.co/OkYjL4xDN7 pic.twitter.com/h1l4LGzYgf

— Felix Kreuk (@FelixKreuk) June 9, 2023

What is Meta MusicGen?

Meta MusicGen, built on the foundation of a powerful Transformer model, follows in the footsteps of language models like ChatGPT, employing cutting-edge AI technology to predict and generate music segments. Just as a language model anticipates the next letters in a sentence, MusicGen predicts the next musical segment based on a given piece of music.

To accomplish this feat, researchers at Meta utilize the EnCodec audio tokenizer, which dissects audio data into smaller units for efficient processing. The brilliance of MusicGen lies in its ability to handle both textual descriptions and musical cues simultaneously, resulting in a seamless fusion of artistic expression.

Training MusicGen involved the use of a vast dataset consisting of 20,000 hours of authorized music. The team leveraged an internal collection of 10,000 high-quality audio recordings, supplemented by music data from reputable sources such as Shutterstock and Pond5. This meticulous training process ensures that MusicGen possesses the prowess to create music that resonates with listeners.

Music generation at its best

While various AI models have excelled in text generation, voice synthesis, and visual creation, high-quality music generation has remained relatively scarce. Music's intricate compositions, overlapping instruments, and the need to capture the complete frequency spectrum pose significant challenges.

However, MusicGen tackles these hurdles head-on, paving the way for an unprecedented musical experience.

How to use Meta MusicGen?

Users eager to explore the capabilities of MusicGen can access it through the Hugging Face API. However, depending on the concurrent user load, the music generation process might take some time. Alternatively, for faster results, users have the option to set up their own instance of the model via the Hugging Face website using this link.

Those proficient in coding can even download the code and manually execute it for a more hands-on approach.

Using MusicGen is a straightforward process. Here is how to do it:

1. Visit the Hugging Face website
2. Click on "Spaces" located at the top right corner
3. Search for "MusicGen" in the provided search box
4. Look for the version published by Facebook
5. Enter your desired prompt in the left box
6. Click on "Generate"

How does Meta MusicGen work?

With MusicGen, your description becomes the catalyst for the creation of a 12-second audio segment. For those seeking a more personalized touch, the model also accommodates the inclusion of a reference audio file, which serves as the foundation for a complex melody. By combining the description and the provided melody, MusicGen endeavors to produce music that truly resonates with the user's preferences.

In a head-to-head comparison, MusicGen outshines its competitors, including Google MusicLM, Riffusion, and Musai. The researchers demonstrate MusicGen's superior output by showcasing a sample website featuring these models.

MusicGen offers four different model sizes, ranging from the compact 300 million parameters to the robust 3.3 billion parameters, unlocking the potential for intricate and mesmerizing compositions. To operate the models locally, it is recommended to use a GPU with a minimum of 16GB RAM.

Meta's vision for the Metaverse

MusicGen represents just one of the many ways Meta utilizes AI to create immersive and captivating experiences for its users. As Meta continues to build towards the realization of a shared virtual world called the metaverse, where people can interact, collaborate, and have fun across multiple platforms and devices, MusicGen is set to infuse the metaverse with a newfound sense of entertainment and musical enchantment.

Advertisement