The music and sound landscape

Google's Lyria line is less a single product than the music engine threaded through Google's whole ecosystem. Lyria 2, opened up in 2025, powers the MusicFX generator, the professional Music AI Sandbox, and real-time tools, and it feeds YouTube's Dream Track for Shorts. By 2026 a newer Lyria generation reached the Gemini app, turning text or images into short songs. Every output is watermarked with SynthIDSynthIDGoogle's watermarking system, which embeds an identifying signal in images, audio, video, and text it generates., the same provenanceProvenanceA verifiable record of how a piece of media was made and by whom, increasingly required so that synthetic content can be traced. layer Google says has now marked over 10 billion pieces of media across image, video, audio, and text. Google's real edge here is not raw musical quality but distribution and reach through YouTube, though its consumer clips have tended to stay short.

The clearest sign of how the field is consolidating is what became of Riffusion. The project that began as a viral experiment generating music from spectrogram images had rebranded to ProducerAI, a browser-based tool that assembles full songs and even custom instruments, and in February 2026 Google acquired it and folded the team into Labs and DeepMind, with ProducerAI running on a preview of the newer Lyria. In roughly three years, a scrappy open experiment became an acquisition target for the company building the platform's music engine.

Stability's Stable Audio is the purest licensed-data play. Stable Audio 2.5, released in September 2025, was pitched explicitly at enterprise sound production, trained on a fully licensed dataset with legal indemnificationIndemnificationA vendor's promise to cover a customer's legal costs if the product triggers a lawsuit, offered by some licensed-data model providers as protection against copyright claims on generated output. for business customers and fast enough to render a multi-part track in seconds. Stable Audio 3.0 followed in 2026, generating tracks up to about six minutes and, notably, shipping most of its model tiers as open weightsOpen weightsModels whose trained parameters are publicly downloadable, allowing anyone to run them locally or fine-tune them. Contrasts with closed/API-only models., with the largest reserved for APIAPI (Application Programming Interface)The programmatic endpoint you call to run a hosted model from your own code instead of a web interface. and enterprise use. Stability trained it entirely on licensed dataLicensed dataTraining data used with the permission of and usually payment to its rights holders, the basis for models marketed as commercially clean., supported by partnerships with major labels, and made commercial use free below a revenue threshold. The whole proposition is that you can ship the output without a lawyer's call.

The open research foundation much of the community still builds on is Meta's AudioCraft, which bundles MusicGenMusicGenMeta's open music generation model, which compresses audio into tokens and predicts them in order, producing instruments and vocals together., AudioGen, and the EnCodecEnCodecMeta's neural audio codec, a refinement of residual vector quantization that became the tokenizer under a wave of speech and music models. codec. Its licensing carries a subtle but important catch: the code is MIT-licensed, but the model weightsWeightsThe learned numbers inside a model that encode everything it knows. 'Open weights' means these are downloadable. are released under a non-commercial license, which makes AudioCraftAudioCraftMeta's open-source audio research bundle, which packages MusicGen, AudioGen, and the EnCodec codec. Its weights are released under a non-commercial license. a research and prototyping tool rather than something you can ship a commercial product on. The open frontier has since moved partly to China. Tencent AI Lab open-sourced its SongGeneration model, and its 2026 successor is a roughly four-billion-parameterParameterOne individual weight. Model size is quoted in parameters, in the millions or billions. More parameters can mean more capability and more cost. model that generates complete songs several minutes long, alongside other open efforts such as ACE-Step. Music generation is not a Western-only race, and some of the strongest fully open models now come from Chinese labs.

Sound effects are the quiet workhorse of this field. ElevenLabs, Stability, and others generate effects from a text promptPromptThe text description you provide to a model to specify what you want it to generate. and return several candidates to choose from, and the more useful variant for film is video-to-sound: point the model at a clipCLIPA text encoder developed by OpenAI in 2021 that learns to align text and images in a shared embedding space. Foundation of most text-to-image models from 2022 onward. and have it generate foleyFoleyThe everyday sound effects of a scene, footsteps, cloth, impacts, now increasingly generated from the video itself. timed to the on-screen action, so an impact lands on the exact frame of contact. ElevenLabs shipped a video-to-sound feature that analyzes a clip and lays in matching effects, and extended its sound-effects model toward longer clips with seamless looping. On the open side, research systems like MMAudio tackle the same video-to-audioVideo-to-audioGenerating a soundtrack timed to an existing video clip, so effects land on the exact frame of the on-screen action. Also called video-to-sound. synchronization problem. This is exactly the problem the video models are now solving natively, which is why dedicated foley tools are converging with, and competing directly against, native audioNative audioSound generated jointly with video in a single pass, so footsteps, dialogue, and ambience line up with the picture, rather than being added afterward. in video.