zoom_in Click to enlarge

AudioLM

▲ ▼

description AudioLM Overview

AudioLM is a framework for audio generation introduced by Google Research in 2022. It models both speech and music by first converting raw audio into discrete tokens using a neural audio codec, then applying a transformer-based language model to predict token sequences in a hierarchical, multi-scale fashion. The system generates long-form, coherent audio continuations without requiring symbolic representations such as MIDI, and was demonstrated on both piano music and spoken-language continuation tasks.

help AudioLM FAQ

Who developed AudioLM?

AudioLM was developed by researchers at Google Research and presented in a paper published in 2022. The framework was designed to demonstrate that high-quality audio generation could be achieved using language modeling techniques applied directly to audio tokens.

How does AudioLM generate audio?

AudioLM works by first converting raw audio into discrete tokens using a neural audio codec, then applying a transformer-based language model to predict sequences of these tokens in a hierarchical manner. This approach allows it to generate both speech and instrumental music directly in the audio domain without relying on spectrogram representations or symbolic notation like MIDI.

Can AudioLM clone a specific person's voice?

AudioLM can generate speech that maintains the acoustic characteristics of a given speaker from a short conditioning clip. However, Google Research has emphasized the framework's research nature and acknowledged concerns about potential misuse, including the generation of convincing deepfake audio, as an area requiring careful ethical consideration.

swap_horiz

Looking for AudioLM alternatives? Compare top competitors ranked & scored

arrow_forward