Definition
Prosody is the pattern of pitch, rhythm, stress, pace, loudness and pausing that shapes how speech sounds beyond its words. It signals questions, emphasis, emotion and when a speaker is about to stop.
How prosody works
The same words can mean different things depending on how they are said. “You’re coming” rises into a question or falls into a statement. Stress moves the focus of a sentence: “I didn’t say he took it” changes meaning with each word emphasized. Pace and pauses mark clause boundaries, and a slower, quieter delivery reads as apologetic or serious.
In speech synthesis, prosody is one of the explicit problems to solve. The W3C specification for SSML lists prosody analysis as a processing step and defines markup that lets an author influence pitch, speaking rate, volume and pauses. It also warns that mixing author-set prosody with what the processor decides itself can interact in processor-specific ways, so results differ between systems.
Neural speech models learn much of this on their own from recorded speech, so a sentence read from plain text already carries plausible stress and rhythm. Controls then adjust it. Many services accept a speaking rate, a volume or an emotion setting per request, and some let the LLM that wrote the reply pick them turn by turn.
Why it matters for voice agents
Prosody is what makes a reply sound like it was meant. A flat reading of an apology, or a rising tone on a statement of fact, can make a correct answer sound dismissive or uncertain. Numbers need rhythm too: a card number read in groups with short pauses is much easier to follow and write down than one read as an unbroken string.
It also works in the other direction. People use pitch and pausing to show they have finished or are about to continue, and a good turn-taking system can use those cues rather than silence alone.
Prosody is judged by ear. A listener rating of naturalness, such as the mean opinion score, is the standard check.
Example
An illustrative instruction to a voice: read “Your balance is 1,204 dollars” at a calm pace, pause briefly after “balance is”, and lower the pitch on the final word to signal that the sentence has ended.