What is the difference between WebRTC and WebSocket?

A WebSocket is one long-lived, two-way connection over TCP. Every message arrives, in order, and the application decides what the bytes mean. WebRTC is a set of protocols for real-time media between peers. It usually runs over UDP and includes codecs, jitter buffers, packet loss recovery, encryption, and NAT traversal. WebSocket moves data; WebRTC moves audio and video.

Is WebRTC faster than WebSocket?

On a clean network, the two have similar latency. The difference shows up when packets are lost. TCP holds back everything behind a missing packet until it is resent, so a WebSocket audio stream stalls. WebRTC skips or conceals the lost audio and keeps playing, which is why it holds up better on Wi-Fi and mobile networks.

Does WebRTC use WebSocket?

Often, but only for setup. WebRTC doesn't define how two peers find each other and exchange connection details, a step called signaling. Many applications send signaling messages over a WebSocket. The audio itself then flows over WebRTC's own transport, not the WebSocket.

Is WebRTC UDP or TCP?

WebRTC media normally travels over UDP, encrypted with SRTP. When a firewall blocks UDP, WebRTC can fall back to TCP or TLS through a TURN relay server. That fallback works, but it brings back some of TCP's stalls under packet loss.

Should a voice agent use WebRTC or WebSocket?

Use WebRTC between the user's device and your servers when users talk from browsers or mobile apps on networks you don't control. Use WebSocket between servers, for example from your agent to speech to text and text to speech APIs, and for clients on reliable networks. Phone calls use neither on the caller's side; they arrive over SIP or a telephony provider's media stream.

Does Cartesia support WebRTC?

Cartesia's real-time APIs for text to speech, speech to text, and Managed Agents use WebSocket. For a WebRTC front end, use LiveKit or Pipecat with Daily. Both have Cartesia plugins that connect to Cartesia over WebSocket from the server, while the user's audio travels over WebRTC.

WebRTC vs WebSocket for real-time voice AI

WebSocket and WebRTC both keep a connection open so audio can flow both ways without a new request for every chunk. The difference is what happens when the network misbehaves. A WebSocket runs over TCP, so every byte arrives in order, even if that means waiting. WebRTC usually runs over UDP and is built for media, so it drops or conceals a late packet and keeps playing.

For a voice agent, that difference decides where each one belongs. WebRTC is for the hop between a person’s device and your servers, where Wi-Fi and cell networks lose packets. WebSocket is for the hops between servers, where they don’t, and for anything that needs a simple, ordered stream of messages. Most production voice agents use both.

This guide is for engineers choosing a transport for a voice agent or a real-time speech feature. It covers how each protocol works, the comparison that matters for audio, and the architectures teams actually ship.

WebRTC vs WebSocket at a glance

	WebSocket	WebRTC
What it is	A two-way message channel	A stack of protocols for real-time media
Transport	TCP (with TLS for `wss://`)	Usually UDP; TCP or TLS through a TURN relay as a fallback
Delivery	Every message, in order	Media packets can be dropped; late audio is concealed
Under packet loss	The stream stalls until the lost data is resent	Audio keeps playing, with brief artifacts
Audio handling	None. You choose the format, buffer, and play it	Opus codec, jitter buffer, loss concealment built in
Connection setup	One HTTP upgrade request	Signaling, ICE candidates, often STUN and TURN servers
Server side	Any web server or language	Usually a media server or a hosted platform
Best for	Server to server, APIs, text and control messages	Browser and mobile audio on networks you don’t control

How a WebSocket works

A WebSocket starts as an ordinary HTTP request with an Upgrade header. If the server agrees, the same TCP connection stays open and either side can send messages at any time. The protocol is defined in RFC 6455, and every browser has a WebSocket API.

WebSocket doesn’t know anything about audio. A message is a text or binary frame, and the application decides what it contains. Speech APIs usually send JSON events with base64-encoded PCM, or raw binary frames. Your code captures the microphone, encodes the chunks, sends them, and on the other end queues the returned audio for playback.

That simplicity is the appeal. A WebSocket goes through most proxies and load balancers, works from any language, and is easy to debug: you can read the messages. The cost is TCP’s delivery guarantee. When a packet is lost, TCP retransmits it, and nothing behind it reaches the application until it does. For a file download that’s exactly right. For live speech, a 200 millisecond retransmit means 200 milliseconds of silence, and then a burst of audio arriving all at once.

How WebRTC works

WebRTC is a set of standards (the overview is RFC 8825) for sending audio, video, and data between peers in real time. The browser handles most of the media work:

Codecs. Browsers must support Opus and G.711 for audio (RFC 7874). Opus compresses speech well and adapts its bitrate to the network.
Transport. Media goes over SRTP (RFC 3711), usually on UDP, with keys negotiated over DTLS. Encryption is mandatory.
Jitter buffer and loss concealment. The receiver holds a few tens of milliseconds of audio to smooth out uneven arrival, and fills small gaps instead of stopping.
NAT traversal. ICE tries direct paths first, uses STUN to discover public addresses, and falls back to a TURN relay when nothing else gets through.
Echo cancellation and noise suppression. Browsers apply them to WebRTC calls by default.

What WebRTC doesn’t define is signaling: how two peers exchange the session description and network candidates before media can flow. You build that part yourself, and it’s very often a WebSocket. That’s why “WebRTC vs WebSocket” is a slightly odd matchup. Many WebRTC apps contain a WebSocket.

The other catch is the server side. WebRTC was designed for browser-to-browser calls. When the other “peer” is your voice agent, you need a server that speaks WebRTC: a media server (an SFU) such as LiveKit, or a hosted platform such as Daily. Running your own TURN servers and media servers is real infrastructure work.

Which one matters for voice AI

A voice agent has at least three network hops, and they don’t need the same transport.

The user’s device to your servers. This is the hop over home Wi-Fi, office networks, and phones on a train. Packet loss and jitter are normal here. WebRTC was built for this hop: audio keeps flowing through a lost packet, Opus lowers its bitrate when bandwidth drops, and the browser cancels the agent’s voice from the user’s microphone. If your users talk to an agent from a browser or a mobile app, this is where WebRTC earns its setup cost.

Your servers to speech models. Your agent streams audio to a speech to text model and text to a text to speech model. These connections run between data centers, where packet loss is rare, so TCP’s ordered delivery costs almost nothing. You also want what WebSocket gives you: ordered text and control events alongside the audio, request contexts, and simple authentication. This is why speech APIs, Cartesia’s included, stream over WebSocket.

Phone calls. A caller on a phone line reaches you over the phone network, then SIP and RTP, or a provider’s media stream. Twilio Media Streams, for example, forward call audio to your server over a WebSocket as 8 kHz μ-law. Neither WebRTC nor WebSocket is on the caller’s side of the call. If you’re choosing a transport for phone agents, read about SIP trunking instead.

So the useful question is where the unreliable network is. Put WebRTC on that hop, and use WebSocket everywhere else.

When a WebSocket is enough on the client

WebRTC isn’t required for every browser voice feature. A WebSocket straight from the client is a reasonable choice when:

The network is predictable. Internal tools, kiosks, desktop apps on office networks, and demos.
You’re prototyping. One connection, no media server, no TURN, and you can log every event.
The audio isn’t conversational. Streaming text to speech for read-aloud, or dictation where a short stall doesn’t break a conversation.
The client is a server. A backend or an edge function that relays audio from somewhere else.

If you go this route, send small audio chunks, keep a short playback buffer, and stop playback immediately when the user interrupts. Test on a lossy network before you ship. Browser developer tools can throttle bandwidth and latency but not drop packets, so use an OS-level network conditioner or a phone hotspot.

Architectures that work

WebRTC front end, WebSocket back end. The most common production setup. The user’s browser or app joins a WebRTC room on LiveKit or Daily. Your agent runs on a server, joins the same room, and streams to speech models over WebSocket. LiveKit Agents and Pipecat both work this way, and both of their Cartesia plugins open a WebSocket to Cartesia from the server. LiveKit and Daily, the company behind Pipecat, are both Cartesia customers. Daily’s CEO, Kwindla Hultman, put it this way: “Cartesia Sonic is the best voice model today for real-time multimodal use cases.”

WebSocket end to end. The browser streams audio to your server over a WebSocket, and your server talks to speech models over WebSocket too. It’s the simplest stack and works well for prototypes and controlled networks. Cartesia Managed Agents also take a WebSocket connection directly, so a trusted server or a browser with a short-lived access token can talk to a hosted agent without running any agent code.

Phone in, WebSocket inside. For phone agents, the call arrives over SIP or a telephony provider, and the agent’s connections to speech models stay on WebSocket. Managed Agents handle the phone side for you, through Cartesia numbers, Twilio, or your own SIP trunk.

Streaming audio to Cartesia over WebSocket

Cartesia’s real-time APIs stream over WebSocket:

Text to speech: wss://api.cartesia.ai/tts/websocket takes text as it arrives from your LLM and returns audio chunks. One connection can carry several contexts, and continuations let Sonic speak a sentence before the LLM finishes the reply.
Speech to text: Ink-2 transcribes a stream of raw PCM over WebSocket and handles turn detection natively.
Managed Agents: wss://api.cartesia.ai/v1/agents/websocket/{agent_id} connects an application to a hosted agent. You send audio_input events with base64 audio in 20 to 100 millisecond chunks and play back audio_output events. When the user starts talking over the agent, Cartesia sends audio_output_clear, and your client stops playback.

The agent WebSocket takes headerless mono audio in pcm_16000, pcm_24000, pcm_44100, or mulaw_8000, and returns audio in the same format:

{
	"type": "session_create",
	"audio": {
		"input_format": "pcm_16000",
		"output_delivery": "speaking_pace"
	}
}

From a browser, never put an API key in client code. Browsers can’t set WebSocket headers, so your server mints a short-lived access token and the client passes it as a query parameter. The WebSocket API guide has a complete session example.

To try it, create an agent in the Cartesia playground, or get an API key and stream your first audio from the text to speech quickstart.