The voice landscape beyond ElevenLabs

The defining goal here is speech-to-speechSpeech-to-speechConverting a recorded performance into another voice while keeping the original timing and emotion, the approach serious film work prefers over pure text-to-speech. in one model with almost no delay, so an agent can listen, think, and answer without the dead air of a transcribe-then-generate pipeline. Cartesia is the purest bet: founded by the Stanford researchers behind the state-space-model line that produced Mamba, it uses that architecture instead of transformers precisely because it scales more cheaply for streaming audio, and its Sonic models target sub-200-millisecond latencyLatencyThe delay between asking for a generation and getting the first result. Distinct from throughput. for phone and agent use. OpenAI treats voice as a native modality of its GPT-4o generation, with a Realtime APIAPI (Application Programming Interface)The programmatic endpoint you call to run a hosted model from your own code instead of a web interface. and Advanced Voice Mode that preserve tone and interruptions in a single model. Google does the same with Gemini native audioNative audioSound generated jointly with video in a single pass, so footsteps, dialogue, and ambience line up with the picture, rather than being added afterward., watermarked with SynthIDSynthIDGoogle's watermarking system, which embeds an identifying signal in images, audio, video, and text it generates., and showcased in NotebookLM's two-host podcast feature. Hume comes at it from emotion science, building a voice interface that reads and adapts to the speaker's emotional state. And Sesame's demo voices went viral in early 2025 for sounding genuinely present in conversation, alongside open base weightsWeightsThe learned numbers inside a model that encode everything it knows. 'Open weights' means these are downloadable..

A surprising amount of the best voice work is open-weight and runnable on your own hardware. Kokoro is the standout for size: an 82-million-parameterParameterOne individual weight. Model size is quoted in parameters, in the millions or billions. More parameters can mean more capability and more cost. Apache-2.0 model, small enough to run on a CPU, that topped a community listening arenaArenaA public leaderboard that ranks models from large numbers of blind pairwise human preference votes, which tends to reward average appeal over fitness for a specific job. at launch and beat models many times larger. Chatterbox, an MIT-licensed model from Resemble AI, clones from about five seconds of audio, adds a rare emotion-exaggeration control, and ships with watermarkingWatermarkingHiding a detectable signal inside generated media so its origin can later be identified. For audio and AI media it remains an arms race rather than a guarantee. baked in. Fish Speech and its expressive OpenAudio line bring actor-like delivery markers under a permissive license. XTTS from the now-wound-down Coqui was long the default open zero-shot cloner across many languages, and MaskGCT represents the masked-generative research branch.

Some of the most useful voice tools are not standalone generators at all but synthesis embedded in a workflow. Descript's Overdub clones your own voice from about ten minutes of consented audio so you can fix a flubbed line by editing the transcript, no re-recording session needed, which is a perfect fit for podcasts and talking-head video. Adobe's contribution is less generation than cleanup and provenanceProvenanceA verifiable record of how a piece of media was made and by whom, increasingly required so that synthetic content can be traced.: its Enhance Speech tool separates voice from noise and room better than almost anything, and it pushes C2PAC2PA (Content Credentials)An open provenance standard, backed by Adobe, OpenAI, and Microsoft, that attaches cryptographically signed metadata proving a file's origin, though the metadata strips easily when a file is re-encoded. Content Credentials across its tools. On the enterprise side, WellSaid Labs and Murf sell clean, consistent, ethically sourced narration for corporate training and e-learning, deliberately not offering open cloning, while PlayHT and Speechify chase the creator and accessibility markets. Resemble AI is the interesting hybrid: it sells cloning and also ships deepfakeDeepfakeA synthetic image, audio, or video of a real person or event made to look genuine, a category the EU AI Act now requires be disclosed. detection and watermarkingWatermarkingHiding a detectable signal inside generated media so its origin can later be identified. For audio and AI media it remains an arms race rather than a guarantee., running both the generation and the anti-generation business at once.

Here is the thing most people get wrong about AI voice in serious film work: the good stuff usually is not text-to-speechText-to-speech (TTS)Generating spoken audio from written text. Modern neural TTS learns the mapping from text to sound directly from data rather than gluing together recorded snippets. at all. It is speech-to-speechSpeech-to-speechConverting a recorded performance into another voice while keeping the original timing and emotion, the approach serious film work prefers over pure text-to-speech. conversion. A real actor performs the line, with all the timing and emotion that pure synthesis still cannot direct reliably, and the tool swaps only the voice identity. Respeecher built its whole business on this for Hollywood, and it is how a young Luke Skywalker could speak in Star Wars-era dialogue and how The Brutalist refined its Hungarian. The performance stays human; the voice changes. That is a fundamentally different and safer proposition than typing a script and hoping.

The other big filmmaking job is dubbing, and it is now a pipeline: transcribe, translate, adapt the script so it fits the timing and the lips, synthesize or convert the voice in the target language, and then, in the most advanced systems like Flawless, alter the on-screen mouth movements to match, because viseme-level lip accuracy is exactly what an audience consciously notices. Vendors report localization costs falling seventy to ninety percent and turnaround shrinking from weeks to days, and 2025 saw early fully AI-dubbed features aimed at US theaters. Underneath all of it sits one unavoidable tradeoff triangle: latencyLatencyThe delay between asking for a generation and getting the first result. Distinct from throughput. versus quality versus control. Real-time systems optimize latency for conversation; film production throws latency away, runs offline, and buys maximum quality and fine control over emotion, timing, and pronunciation instead. That is why studios reach for speech-to-speech and heavy post rather than instant TTS, and why the same company that wins voice agents may not be the one that wins the edit bay.