IVR testing checks that an interactive voice response system answers calls, plays the right prompts, understands keypresses and speech, and sends each caller to the right place. It covers single paths (functional tests), every path after a change (regression), many calls at once (load), and how the call sounds and feels to the caller (experience). A good test calls the real phone number, because most failures happen between components rather than inside one.

Can IVR testing be automated?

Yes. An automated test places a real call, sends keypresses or plays recorded speech, and checks what the system says and where the call ends up. Keypad menus are easy to automate because the expected prompt is fixed. Conversational IVRs and AI voice agents need a test caller that can respond to whatever the system says, plus a check on the outcome rather than the exact wording.

What is the difference between IVR load testing and stress testing?

Load testing runs the call volume you expect at peak and checks that answers, prompts, and transfers still work. Stress testing pushes past that volume until something breaks, so you learn which component fails first and whether it fails safely. Soak testing holds a normal load for hours to find slow leaks, such as connections that never close.

How do you test an AI voice agent?

Write scenarios with an expected outcome, not an expected script: the caller wants to reschedule, so the appointment should move and the caller should hear the new time. Run each scenario several times with different phrasing, accents, background noise, and interruptions. Score the outcome with a rule or an LLM judge that reads the transcript, and measure the delay from the end of the caller's turn to the start of the reply on the phone.

How often should you test an IVR?

Run the regression suite before every change to prompts, routing rules, models, or telephony, and keep a small set of monitoring calls running in production every day. Changes you didn't make count too: a carrier route, a backend API, or a model update can break a path that passed last week.

Does Cartesia offer IVR testing?

Cartesia builds voice agents and the speech models behind them, not a dedicated test suite. Managed Agents scores every completed call with built-in metrics and custom LLM-judge metrics you define, and an agent can dial a phone menu and press keys, so you can build a test caller. For large-scale load testing or a managed test service, use a testing vendor such as Cekura, Cyara, Coval, or Hamming.

IVR testing: types, tools, and voice agent tests

IVR testing means calling your phone system the way a customer would and checking that each call ends in the right place. Most broken IVRs fail at the seams: a prompt that changed while the routing rule didn’t, a transfer to a queue that closed at 6 p.m., a speech model that hears “pharmacy” as “for Marcy”. You only see these failures by placing real calls through the real phone path.

This guide covers the standard test types for keypad and speech IVRs, then what changes when the thing answering the phone is an AI voice agent. The short version: a keypad menu has one correct script per path, while a voice agent has many acceptable conversations and one correct outcome. Your tests need to check the outcome.

The types of IVR testing

Test	What it answers	When to run it
Functional	Does this path play the right prompts and end at the right destination?	When you build or change a path
Regression	Does every path still work after a change?	Before every release
Speech recognition	Does the system understand the words callers actually use?	When you add an intent, a language, or a new caller group
Load	Does it keep working at expected peak volume?	Before launch and seasonal peaks
Stress	What breaks first above peak, and does it fail safely?	Before launch, after infrastructure changes
Soak	Does anything degrade over hours of normal traffic?	Before launch
Experience	How long do callers wait, and does it sound right?	Continuously, from production calls
Failover	What happens when a backend, queue, or carrier is down?	Before launch, then quarterly

Functional and regression tests catch most problems. Load, stress, and soak tests matter when the IVR sits in front of a contact center with real peaks, such as open enrollment, a product recall, or the Monday after a billing run.

A traditional IVR is a tree. Every node has a prompt and a fixed set of inputs, so every path can be written down and checked.

Start by exporting the call flow and turning each path into a test case. A path is the input sequence plus the expected result: press 2, then 1, then enter an account number, and the call should reach the billing queue with the account attached. Include the paths callers take by accident, not only the happy ones:

No input at all, until the timeout
An invalid key, then a valid one
Pressing 0 or saying “agent” at every level
Calling outside business hours
An account number that doesn’t exist
Entering digits before the prompt finishes (callers who know the menu do this)

For speech menus, the test inputs are recordings. Use different speakers, a phone line instead of a studio microphone, and background noise such as a car or a kitchen. The words should be the ones customers say in your call logs, not the labels in the menu: callers ask about “my bill” far more often than they say “account services”.

Check the data along with the audio. A test passes when the call reaches the billing queue and the agent’s screen shows the right account. If the IVR collected the number but the transfer dropped it, the caller has to read it out again, and that’s a failure even though every prompt played correctly.

Testing a conversational IVR or AI voice agent

When an AI voice agent answers the phone, the script-matching approach stops working. The agent can say “Sure, let me move that for you” on one call and “Okay, I can change that appointment” on the next. Both are fine. A test that compares exact prompt text fails on every run and tells you nothing.

Write scenarios with an expected outcome instead:

Scenario	Caller behavior	Pass condition
Reschedule	Gives name and date of birth, asks for Thursday afternoon	Appointment moved in the scheduling system; caller hears the new time
Wrong identity	Gives a date of birth that doesn’t match	No appointment details read out; caller offered another way to verify
Mind change	Asks about billing, then says “actually, it’s about delivery”	Handles delivery; doesn’t transfer to billing
Interruption	Starts talking while the agent is mid-sentence	Agent stops speaking and responds to the new request
Person, now	Says “let me talk to a human” in the first sentence	Transfers, or gives the after-hours option, without arguing
Backend down	Scheduling API returns an error	Agent doesn’t claim the booking succeeded
Number readback	Gives a 10-digit order number with pauses	Agent repeats every digit correctly

Run each scenario more than once. A voice agent can pass a scenario four times and fail it on the fifth because the caller phrased the request differently or the model took a different route through the conversation. Five to ten runs per scenario, with varied phrasing, catches most of that. Personas help: a hurried caller, a confused caller, a caller with a strong regional accent, a caller in a noisy room.

Score each run in two ways. Hard checks read your systems: did the appointment move, did the transfer reach the right number, did the agent end the call? Judgment checks read the transcript: did the agent confirm the new time out loud, did it stay polite when the caller got frustrated? An LLM judge with one narrow question per metric works well for the second kind. A judge asked “was this a good call?” returns opinions; a judge asked “did the caller hear the new appointment time before the call ended? Answer yes or no and quote the line” returns something you can count.

The conversational IVR guide has a routing-specific test set, including how to read a routing accuracy number without overtrusting it.

Measure latency on the phone, not in the logs

Callers hear delay as the gap between the moment they stop talking and the moment the agent starts. That gap includes deciding the turn has ended, the LLM, speech synthesis, and the trip back through the telephony provider and the carrier. A dashboard that shows only the text-to-speech time to first byte covers one part of it.

Measure from a recording of the call itself. Record both sides, mark the end of each caller turn and the start of each reply, and report the median and the slowest 5%. The slowest calls are the ones that get “hello? are you there?” and a repeated question.

Outbound calling has a hard limit of its own. Under the TCPA’s abandoned-call rule, a call counts as abandoned if the person who answers isn’t connected within two seconds of finishing their greeting. 2X Solutions cut response latency from 500–900 ms to under 100 ms and brought its abandoned-call rate from 3.4% to under 1.25% across millions of calls a month. If your agent places calls, test the time from “Hello?” to the agent’s first word on every outbound scenario.

Regression testing when models change

Voice agents change in ways keypad menus never did. A new prompt, a new LLM, a new voice, or a new speech model can each change behavior on paths you didn’t touch. Treat each of these as a release and rerun the full scenario set.

Pin what you can. Cartesia’s Sonic has dated snapshots such as sonic-3.6-2026-08-27 that never change, so you can run your scenarios against a new snapshot before switching production to it. Managed Agents saves every configuration change as a version, so if a new prompt fails a scenario that passed before, you can restore the previous version and keep investigating.

Keep the failures. Every production call that went wrong becomes a new regression scenario, with the caller’s real phrasing. After a few months, that set is more useful than anything you could write up front.

Building a test caller with Cartesia

You can build a basic automated test caller on Cartesia Managed Agents itself:

Create a test agent whose instructions describe one scenario: who it is, what it wants, and how it should react to the IVR.
Turn on the send_dtmf system tool so the agent can press keys while it navigates a phone menu.
Place an outbound call from the test agent to your IVR’s number. For a full suite, batch calling queues up to 5,000 calls and dials them up to a concurrency limit you set.
Attach custom metrics to the test agent, each one a narrow LLM-judge question such as “Did the call reach the pharmacy queue?” Then export the results as CSV.

The same metrics work on your production agent. Managed Agents scores every completed call with two built-in metrics: whether the call ended without an agent failure, and the time to the first byte of audio on the agent’s first reply. Any custom metrics you assign run on later calls, which turns production traffic into a monitoring set.

This setup suits regression runs and production monitoring. Batch calls share your account’s agent-call concurrency limit with production traffic, so a load test at contact-center peak needs its own capacity plan. If you want a managed test suite or a team that runs load tests for you, use a dedicated vendor. Cekura runs simulated callers against voice agents before and after launch and can use Cartesia voices for those callers. Cyara is the long-standing option for classic contact-center IVRs. Coval and Hamming also test AI voice agents.

Where to start

If you test nothing today, start with eight calls: the four most common call reasons, an after-hours call, a caller who asks for a person, an invalid input, and one call while a backend is down. Run them by hand through the real number once, write down what happened, then automate the same eight. Add scenarios from real failures as they come in.

If you’re replacing a keypad menu with a voice agent, run both against the same scenario set before you switch. The contact center automation guide covers the rollout, and the voice agent guide explains the parts you’re testing. To try an agent on your own call flow, create one in the playground and call it.