I think Coval's streaming STT benchmark is useful because it puts the inference stack back into the conversation, where conversational products actually live. If you only compare model weights, you miss the part of latency that users feel most: the deployment path, the region, and the endpoint you put behind the model.
What Coval's streaming STT benchmark actually measures
As checked on September 14, 2026, Coval's WER page lists 30 STT models and 28 TTS models over a rolling 30-day window. The STT table reports word error rate alongside transcript latency and sample counts. Some rows have tens of thousands of samples; others have far fewer. Those differences matter when interpreting a ranking. Coval WER leaderboard
The page also exposes more than one latency view, including Time to Final Segment, Time to First Token, and Time to Final Transcript. That is useful because different products care about different moments: a live captioning tool cares about final transcript speed, while a voice agent cares about when it can safely start speaking again. Coval benchmarks overview Coval WER leaderboard
Coval runs the public benchmark from a fixed region. That makes the comparison cleaner, because every model is being hit from the same place, but it also means the number is a benchmark number, not the latency your users will see from their own cities or networks. Coval benchmarks overview
That setup matters because the leaderboard measures the inference leg end to end. If a model sits behind a shared endpoint or a badly placed deployment, the numbers absorb that cost. In other words, the benchmark is not just scoring the weights, it is scoring the path the audio takes to get to and from them. Coval WER leaderboard Coval benchmarks overview
Snapshot: where Baseten sits on Coval's STT leaderboard
In the Coval table checked on September 14, Baseten's Qwen3 ASR 1.7b endpoint records 39 ms TTFS and 3.9% WER across 1,192 samples. These are measurements of that endpoint under Coval's test conditions. Coval WER leaderboard
Baseten's own post says the deployment is approximately 5x faster than OpenAI's. The raw Coval rows are more precise than that headline, and they also show why the exact ratio depends on which OpenAI endpoint you compare against, since the OpenAI rows on the board do not all have the same TTFS. Baseten's benchmark post Coval WER leaderboard
Baseten's September 9 post describes its early-access result as approximately five times faster than OpenAI's. Treat that as a dated vendor claim with a specific baseline. Before applying the ratio to a purchasing decision, check the model, endpoint, measurement date, and latency definition on both sides. Baseten's benchmark post Coval WER leaderboard
How dedicated inference and deployment choices drive those gaps
Coval annotates some rows as dedicated inference and others as shared. That is the part many model charts leave out, and it is also the part that moves a deployment left on the Pareto plot without changing the model weights at all. Coval WER leaderboard
Baseten's post points to dedicated inference and tuned autoscaling as the levers behind its result. That lines up with how production systems behave: reserved capacity cuts noisy-neighbour variance, and better placement reduces the network time before the model even starts working. Baseten's benchmark post
A shared endpoint can look fine in a demo and still feel worse in production, because latency consistency matters as much as the median when the system is under load. If p95 drifts or the p25-p75 band widens, the user experiences hesitation, not just a slower benchmark row. Baseten's benchmark post Coval WER leaderboard
That is why co-location matters. Putting inference near the user cuts round-trip time, and in conversational products that reduction is visible even when the model itself has not changed. Coval benchmarks overview
What the benchmark controls, and where it can mislead product decisions
Coval's fixed region and fixed runner are a good way to remove one source of noise. They make the benchmark more reproducible, but they also make it easier to confuse benchmark fairness with product reality, which is a different thing entirely. Coval benchmarks overview
The sample counts matter for the same reason. Coval shows counts on the rows, and some are large while others are much smaller, which tells you when a point estimate has the weight of tens of thousands of utterances behind it and when it does not. Coval WER leaderboard
That is one reason vendor comparisons that ignore the inference tier are so often misleading. A model can be easy to host in a lab and still be expensive or inconsistent once you add orchestration, autoscaling, and the network between the caller and the server. Baseten's benchmark post Coval benchmarks overview
The methodology also changes how you compare one leaderboard to another. TTFS, TTFT, and final transcript timing are not interchangeable, and if you read them as if they were, you will end up comparing numbers that answer different questions. Coval WER leaderboard
Build: how to reproduce a low-latency, low-WER STT pipeline (a Cartesia Ink-2 example)
If you are trying to make a voice agent respond faster, start by choosing an STT model that was built for turn-taking, not just transcription. Cartesia's Ink-2 exposes configurable turn detection and keyterm prompting, which means you can tune when the model decides the user is done and also teach it names and jargon it should expect. Cartesia Ink-2 docs
That matters because endpointing is part of responsiveness. If your model waits too long to commit a turn, the rest of the stack sits idle; if it ends too early, you interrupt the user. The right knob is not "more model," it is the turn detector you can actually control. Cartesia Ink-2 docs LiveKit Agents
The same model can land differently depending on how you deploy it. Coval's rows make that visible by separating dedicated inference from shared setups, and Baseten's post shows the same pattern from the provider side: deployment choices move the latency number without requiring a different set of weights. Coval Ink-2 row Baseten's benchmark post
If final transcript accuracy matters more than the earliest partials, you should bias toward semantic finalization. If your product needs to speak first, faster partials may win, but then you are explicitly choosing a bit more transcription risk in exchange for a shorter conversational pause. Artificial Analysis AA-WER Streaming
Artificial Analysis's June 1, 2026 report measured Ink-2 with semantic endpoints at 3.59% WER and 0.21 seconds for the first final transcription after detected end of speech. Different datasets, endpoint settings, and timing definitions can produce different results for the same model. Artificial Analysis AA-WER Streaming Coval Ink-2 row
Keyterm prompting is the other lever worth using before you reach for a new model. If your transcript is mangling product names or internal jargon, telling the STT what to expect is usually cheaper and more targeted than swapping providers. Cartesia Ink-2 docs
For production tuning, framework support matters too. LiveKit Agents exposes provider configuration hooks that are exactly where you want turn-detection and keyterm settings to live, because the point is not to admire the model but to wire it into the call path you actually ship. LiveKit Agents
Practical trade-offs: fastest transcripts often cost accuracy, and vice versa
Artificial Analysis's June 1, 2026 results report Deepgram Flux at 7.36% WER and 0.020 seconds for the first final transcription after detected end of speech. Ink-2 with semantic endpoints recorded 3.59% WER and 0.21 seconds on that measure. Flux returned the final transcript sooner; Ink-2 made fewer transcription errors in this test. These results describe the tested configurations and dataset. Artificial Analysis AA-WER Streaming
That is the right way to think about a streaming STT benchmark. It does not tell you which model is "best" in the abstract, because a model that wins on speed can still lose on errors that matter in a real voice agent, like misrouting a caller or missing a product name. Coval WER leaderboard Artificial Analysis AA-WER Streaming
Tail latency is the part that turns a mostly-good system into an annoying one. A low median with fat p95 or p99 tails means users sometimes wait through the kind of pause that makes the agent feel uncertain, even when the average case looks fine. Coval WER leaderboard
Test deployment placement, endpoint capacity, turn detection, and keyterm prompting separately. Record the effect of each change on errors and latency before combining them. That gives you evidence for which change improves your application. Baseten's benchmark post Cartesia Ink-2 docs
What engineers should measure in their own stack before choosing a provider
The number that matters is the one you measure through your own path. For STT, that means timing from end-of-speech detection to the moment the final segment arrives, under your actual telephony or WebRTC conditions, not in a clean lab path that bypasses your real network. Coval benchmarks overview
You also want percentile data, not just a median. If p50 looks great but p95 and p99 spread out, the user hears a system that is sometimes quick and sometimes awkward, which is usually worse than a slightly slower system that stays consistent. Coval WER leaderboard
Replay representative production audio, including the accents, background noise, and domain vocabulary your application must handle. Keep a separate evaluation set so tuning decisions do not simply optimize for the recordings you used during development.
Use the public leaderboard as a shortlist, not as a final answer. Once you have the shortlist, run it in your own region, under your own concurrency, and through your own stack before you decide what your users will hear. Coval benchmarks overview Baseten's benchmark post
Start with a small set of candidate endpoints and a fixed evaluation corpus. Record transcription errors, finalization delay, and end-of-speech-to-first-audio latency under expected concurrency. Choose the configuration that meets your application's error and response-time targets, then repeat the test when the model or deployment changes.