AudioCraft

AudioCraft is Meta AI's open-source framework for generative AI audio, comprising multiple models for different audio generation tasks. The framework's release as open source has made it one of the most accessible AI audio generation tools, enabling researchers, developers, and creators to generate audio locally without API costs or usage restrictions.

MusicGen is AudioCraft's music generation model, capable of creating music from text descriptions and optionally conditioning on melody references. The model can generate music in specific genres, moods, and tempos as described in natural language, and when provided with a melody reference, can create a new piece that incorporates the specified melodic material.

AudioGen creates sound effects and environmental audio from text descriptions — rain sounds, city ambience, specific sound effects, and other non-musical audio. This sound effect generation capability complements MusicGen to provide comprehensive audio generation across the types of audio used in media production.

AudioCraft's EnCodec is the audio codec model that underlies the framework's ability to compress audio into discrete tokens for model processing. This codec is also available for other researchers to use for audio compression and generation research.

For researchers studying AI audio generation, AudioCraft provides state-of-the-art models with open access to weights, architecture code, and training details. For developers building audio applications, the open-source models enable local deployment without the API costs and data privacy concerns of cloud-based alternatives.