A useful TTS API comparison starts with the endpoint your application will call. A model-family name leaves too much unspecified: the exact model, hosting arrangement, region, voice, and measurement method can all differ between entries on a leaderboard.
Coval's Qwen3 TTS entries illustrate the problem. They share a family name, but the published endpoint descriptions do not establish an experiment in which identical weights run under otherwise identical conditions. The practical lesson is to compare complete services and keep the limits of the evidence visible.
What do the Qwen3 TTS benchmark entries actually compare?
The figures below are the 30-day average TTFA values displayed together on Coval's TTS ranking when viewed on September 15, 2026. The linked model pages supply the hosting and licensing details. These are measurements of separate endpoints, not a controlled test of identical systems.
- Qwen3 TTS Fast, hosted by Nari, reports 71 ms mean TTFA. Coval lists it as open-weight, shared inference in the US.
- Qwen3 TTS 1.7b, hosted by Baseten, reports 106 ms mean TTFA. Its entry describes open-weight, dedicated inference in the US.
- Qwen3 TTS Flash Realtime, hosted by Alibaba Cloud, reports 751 ms mean TTFA. Its entry describes a proprietary official API in Asia.
The distinctions matter as much as the latency figures. Dedicated and shared inference describe different hosting arrangements. The Alibaba entry is a proprietary endpoint in a different region. Calling all three rows "the same weights" would require evidence beyond their names.
These measurements can help shortlist endpoints. They do not isolate how much of a difference comes from the model, network path, hardware, scheduling, or another implementation detail. Even a large gap between rows is an observation about those measured services, not a controlled explanation of its cause.
Which latency number should a TTS API comparison use?
First audio arrival and first audible speech are different milestones. A client can receive a chunk that begins with silence. Coval's published methodology defines TTFA using both the time until the first audio chunk arrives and the leading silence before audible output. That definition is worth recording alongside any number you copy from the board.
A comparison sheet should preserve the statistic too. A mean, median, and p95 answer different questions. If the purchasing decision depends on slow responses, a low average alone does not settle it. Record the measurement window and sample count so that a later reader can tell what the result describes.
For an application test, choose explicit start and stop events. For example, measure from submitting text to the first audible sample played by your client. Keep a separate measurement for connection setup, and record whether each request reused an existing connection. This makes the test useful for your deployment instead of leaving "latency" to mean something different in each row.
How should you compare speech quality?
Coval also reports word error rate for TTS endpoints. Its WER benchmark measures transcription errors in the generated speech. That is a useful signal about intelligibility, but it is not a complete listening evaluation.
Build a small test set from the text your application will actually say. Include names, dates, amounts, abbreviations, and sentences that cross the language or pronunciation boundaries your users encounter. Listen for omitted words and wrong pronunciations, then separately judge pacing, emphasis, and whether the voice fits the interaction.
Keep those judgments separate from speed. A voice that begins promptly but misreads the amount in a payment reminder is a poor choice for that task. Likewise, a quality score that ignores the delay before a reply leaves out part of a conversational experience. Decide which errors would rule out a provider before you look at the results.
How can you make the comparison useful for production?
Use the leaderboard to select a manageable shortlist, then run the same application-level test against each candidate. Keep the input text and output format consistent wherever the APIs allow it. Record the exact model identifier, voice, region, concurrency, and any configuration differences you cannot eliminate.
Test normal traffic and the busiest load you expect to support. Track failed requests and interrupted playback alongside latency, rather than calculating a fast average from successful requests alone. Keep the raw observations so that a model update or configuration change can be compared against the same baseline later.
For a Cartesia candidate, the Sonic 3.6 documentation explains that the sonic-3.6 alias follows stable updates, while a dated model ID stays fixed. Pin a dated version when you need a repeatable baseline. Start by listening to Cartesia voices with your own text in the playground, then evaluate the API in the client you intend to ship. A listening session helps choose a voice; it does not replace the deployment test.
Treat self-hosting as a separate operational decision. Put hardware capacity, deployment work, monitoring, and maintenance into that comparison alongside model inference. An attractive latency result does not by itself answer whether your team should operate the service.
The strongest choice is the endpoint that meets your latency and speech-quality requirements under a documented workload. A shared model-family name is useful for organizing a shortlist. The evidence for the final decision comes from the service you will actually deploy.