Text to speech with emotion: how to direct it

Text to speech with emotion means a synthetic voice that sounds pleased, worried, apologetic, or annoyed when the moment calls for it, instead of reading everything in the same polite monotone. Modern models already do a lot of this on their own: they read the sentence and work out how it should sound. The controls you add on top are for nudging, not for rescuing a line that was written flat.

That ordering is the whole guide in one sentence. If you build voice agents or apps, most of the emotional quality you hear comes from the words your system writes and the voice you pick. Tags and parameters adjust what’s left. Below is how each approach works, how the main APIs expose it, working code for Cartesia’s Sonic, and how to tell whether the result is any good.

Where the emotion in TTS comes from

A sentence doesn’t specify one correct performance. “I can’t believe it” can be delight, fury, grief, or sarcasm. Our research team calls text to speech a one-to-many problem: the model has to choose where to pause, which words to stress, how fast to go, and what feeling to carry, often from text that doesn’t say.

There are four ways a model makes that choice:

• It infers it from the text. The model reads the surrounding words and punctuation and picks a delivery. This is the default on every current model, and the one that matters most.
• You name it. A parameter or tag says angry or calm. The model leans that way, within reason.
• You pick a style. Some voices ship with named speaking styles, such as a customer-service style or a newscast style, each recorded or trained as its own performance.
• You describe it. A natural-language prompt says “say this warmly, like you’re reassuring a friend.” The model interprets the description.

Every API mixes these. What differs is which one it treats as the main control.

How the main APIs take emotion direction

Read from each provider’s documentation on October 12, 2026:

Hume’s Octave, which took acting instructions in plain language, belonged in this table until recently. Hume is shutting down its TTS and EVI APIs on November 13, 2026. If you’re moving off it, our Hume alternatives guide covers what to save first.

One pattern matters for production. Prompt-style controls (Gemini, the OpenAI model that’s leaving) are free text, so there’s no fixed vocabulary to test against: to reproduce a delivery, you keep the exact wording of the instruction and retest whenever it changes. A named value (Sonic’s emotion list, Azure’s styles) is a setting you can version, test, and roll back, which is what you want when 10,000 calls a day go through the same agent.

Write the emotion into the words first

This is the step most teams skip, and it’s the one that works best.

Sonic 3.6 varies its pacing and intonation to match the emotional context of the transcript, with no tags. In blind head-to-head tests in US English, listeners preferred Sonic 3.6 over Eleven v3 92% of the time, and the reason given was better intonation and context-driven emotion. In other words, the model is already listening to your words. Give it words worth listening to.

For a voice agent, that means the emotion lives in the LLM prompt. If you want an empathetic tone, the LLM has to write empathetic text. Compare two replies to a customer whose order is late:

• “Your order is delayed. The new delivery date is Thursday.”
• “I’m sorry, I know you were counting on this. It’s now arriving Thursday.”

No tag will make the first one sound sorry. The second one sounds sorry with no tag at all. Our guide to choosing voice AI models makes the same point from the other side: Sonic emotes in line with the text and disregards a configuration that contradicts it.

A few writing habits that help any model:

• Keep normal punctuation. A question mark, a comma before “but”, and an exclamation point all carry information about delivery. Stripping punctuation to “clean” LLM output makes speech flatter.
• Let the agent hesitate where a person would. Sonic reads in-transcript disfluencies like “uh” and “hmm” as a natural thinking pace. Use them sparingly, before a looked-up number or after a hard question, not in every sentence.
• Don’t narrate the emotion. “She said angrily” gets spoken out loud. Put the anger in what she says.

Pick a voice with the right temperament

Voices have a default mood. A voice designed as a cheerful fixer won’t sound like a trusted confidant no matter what you tag. Match the voice to the job first, then tune.

On Cartesia, the voice library marks expressive voices with an Emotive tag, and emotion controls respond best on those. The docs name Leo, Jace, Kyle, Gavin, Maya, Tessa, Dana, and Marian as the voices with the strongest emotional range. If you clone a voice, the clone carries the delivery of its source recording, so record the sample in the mood you want the agent to have.

Control emotion with Sonic

Once the words and the voice are right, here’s how to push the delivery further. These examples use the Cartesia Python SDK version 4.2.0 and sonic-3.6. Set CARTESIA_API_KEY in your environment first, and install cartesia[websockets] for the streaming example. Our Python text-to-speech tutorial covers setup.

One emotion for the whole request

import os

from cartesia import Cartesia

with Cartesia(api_key=os.environ["CARTESIA_API_KEY"]) as client:
    response = client.tts.generate(
        model_id="sonic-3.6",
        transcript="I'm sorry, I know you were counting on this. It's now arriving Thursday.",
        voice="6ccbfb76-1fc6-48f7-b71d-91ac6298247b",  # Tessa, an Emotive voice
        language="en",
        output_format={"container": "wav", "encoding": "pcm_s16le", "sample_rate": 44100},
        generation_config={"emotion": "sympathetic", "speed": 0.95},
    )
    response.write_to_file("apology.wav")

generation_config applies to that request only; there’s no account-wide setting. speed runs from 0.6 to 1.5 and volume from 0.5 to 2.0, and Sonic treats all three as guidance rather than strict multipliers so the speech stays natural. For a clearly audible change, make a bigger adjustment: 0.7 instead of 0.9.

Change emotion inside a transcript

transcript = (
    '<emotion value="calm"/> I can help with that. '
    '<emotion value="excited"/> Good news, your refund went through this morning!'
)

Inline tags are handy for a scripted line with a clear turn in it. Mid-transcript shifts are the most experimental part of the feature, so for an important shift, send each part in its own generation context instead.

Change emotion while streaming

On the WebSocket API, each chunk of text you push can carry its own settings. This is the pattern for an agent that streams LLM tokens to speech as they arrive:

with client.tts.websocket_connect() as ws:
    ctx = ws.context(
        model_id="sonic-3.6",
        voice="6ccbfb76-1fc6-48f7-b71d-91ac6298247b",
        output_format={"container": "raw", "encoding": "pcm_s16le", "sample_rate": 44100},
        language="en",
    )
    ctx.push("Well, maybe if you just ")
    ctx.push("loosen up a little!", generation_config={"emotion": "angry"})
    ctx.no_more_inputs()
    for response in ctx.receive():
        if response.type == "chunk" and response.audio:
            ...  # play or buffer the audio

That example comes from the WebSocket emotion example in our docs. For a laugh, put [laughter] in the transcript. For an exact pause, use <break time="1s"/>, though a comma or period usually gives a better one.

Which value to choose

Start with the six primary emotions: neutral, calm, angry, content, sad, and scared. They have the most training data behind them. The other 52 (sympathetic, apologetic, curious, skeptical, determined, sarcastic and the rest) are in the full list, and are worth testing when a primary value is close but not quite right. Many support agents get by with three: calm by default, sympathetic for bad news, and content when the problem is solved.

How to tell if the emotion is right

Emotion is the part of TTS quality that a metric won’t catch. A transcript-based check like word error rate scores a furious reading of “Thanks for waiting” as perfect. You have to listen, and you have to listen with the situation in mind.

• Tell the listener the scenario. “A support agent telling a customer their flight is cancelled” gets you a useful judgment. “Rate this clip” gets you a judgment about the voice.
• Test complete conversations, not single lines. An emotion that fits one turn can feel strange three turns later, and a voice that drifts in mood across a call is a failure you only hear in context.
• Expect listeners to disagree. Some people like an expressive agent; others find the same performance overacted and tiring. Report the split instead of averaging it away. Our guide to mean opinion scores covers why one number hides this.
• Listen through the real streaming path. A model that picks the wrong emotion on the first audio chunk sometimes can’t recover. You won’t hear that in an offline render.

Where Cartesia fits

Sonic is built for speech that happens in real time: voice agents, phone calls, and apps that talk back. Emotion comes from the transcript by default, the controls work while streaming, and the emotion setting is a named value you can test and version like the rest of your agent. If you’re building a phone agent and don’t want to wire up speech recognition, an LLM, and TTS yourself, Managed Agents runs the whole loop with Sonic as the voice.

If you’re producing an audiobook or a video voiceover and want to edit a performance line by line on a timeline, a studio editor fits that job better than an API. Our ElevenLabs alternatives guide lists several.

To hear the difference yourself, open the playground, pick an Emotive voice, and read the same line with calm and then sympathetic. Then rewrite the line so it sounds sorry without any tag. The second experiment is usually the more instructive one.

Related guides

• Is this TTS model good?
• What is a mean opinion score (MOS)?
• What is neural TTS? How neural voices work
• Python text to speech: save audio as a WAV file