A text-to-speech (TTS) model generates speech. But the model and the voice it speaks with are not the same thing.
If you know LLMs, a voice is loosely analogous to a system prompt: the model supplies the general capability, and the input shapes what it generates.
In TTS, voice is an input often separate from the model, and it shapes the model’s response. And the model’s response - audio - is much more complex than “mere” text. Models that generate speech had to know what should be said, how it should be said, at what pace, how loud, what sort of intonation, prosody, emphasis…all the many characteristics and metrics for a “good voice”. These are instructions you’d normally want to give the model, but most TTS models allow you to specify one or two inputs - the transcript and a selected voice.
Voice Model Inputs
A fixed speaker model takes a single input - the transcript. The model is trained to speak in a single voice (encoded in the model’s learned weights). It does not take a voice as an input.
So for all transcripts supplied, it returns audio that sounds like the same speaker every time.
> Fixed-speaker model + Transcript = speech in one voice
When the only voice you hear is the one the model ships with, model and voice look like the same thing— you can’t cleanly separate them during normal use. Producing another speaker requires adapting or retraining the model.
However, a multi-speaker model accepts a voice selection as another input. So you provide the transcript and select a voice from a fixed list of voices that the model supports.
> Multi-speaker model + Transcript + Selected Speaker = speech in that voice
The speech synthesis model remains the same, while the selected voice changes the speech characteristics it produces.
Keep the transcript constant, but swap the voice and the speech changes — it sounds like a different speaker.
Voice A
Voice B
Zero-shot cloning accepts a new speaker
A zero-shot voice-cloning model can condition TTS speech to sound like a speaker outside its preset catalog. It accepts a transcript and an audio clip of a reference speaker as the “prompt”.
> Cloning model + Transcript + Speaker Audio clip = Cloned Voice
Since audio contains more information than text, the audio clip “prompt” allows for the most fine-grained control of the synthesized voice. This is what makes it “Zero-shot”, which means the system can handle a speaker it has never been trained or fine-tuned on.
It generalizes from the reference audio clip during voice generation.
This is how Cartesia Instant Voice Cloning works.
You provide a short audio clip of your voice as the “prompt” - you don’t need more than 10s to 60s of input audio to condition the model. Cartesia’s Sonic, which is an end-to-end TTS model that handles cloning as well, extracts your voice’s characteristics from that recording and uses them to “prompt” the TTS model to generate speech that sounds like you.
Professional voice cloning uses a larger recording dataset to fine-tune a model. This does more than just conditioning the model at runtime with a voice sample. It actually updates and adapts the model to the input samples. Unlike IVCs, professional voice cloning takes more time - up to a couple of hours. Interestingly, we’ve observed that Cartesia PVCs are so good that they might copy unwanted parts of the audio: microphone plosives, artifacts from recording equipment, background noise, and so on, which is actually preferred in many use cases!
A recording contains more than identity
Customers ask for voices that sound “grittier,” “warmer,” or “more confident.” A reference clip also contains pacing, pauses, room noise, mouth sounds, and details from that specific performance. The system must work out which traits belong to the speaker and which came from the recording.
Long pauses can affect generated pacing. Background noise may be reduced, ignored, or reproduced. Two models may respond differently to the same source material.
Cartesia’s voice-cloning guidance recommends clean audio, short pauses, and source delivery that matches the target use case.
A focused, measured voice is useful for customer support but can make billing disputes feel dismissive or unempathetic. An upbeat voice can energize a product walkthrough or onboarding assistant. These voices exhibit different speech characteristics but are generated by one model.
“How to choose a voice?” and “Is this TTS model good?” are two separate questions — and mixing them up means “debugging” the wrong thing when your voice agent sounds off.