Conversational AI in healthcare is software that talks with patients to get something done: booking a visit, answering “are you open Saturday?”, collecting intake details, or chasing an insurer for a claim status. Most of those conversations still happen on the phone. Practices keep shipping patient portals and online scheduling, and the phone keeps ringing anyway. Hello Patient’s founder saw this firsthand at Carbon Health, which handled more than 30,000 inbound calls a day.
So the practical question is which patient calls a voice agent should answer, and how to build it so it is safe with patient data. This guide is for the operations, IT, and engineering leads deciding that. Short version: start with administrative calls, draw the clinical line in the agent’s instructions and test it, sign a business associate agreement with every vendor that touches patient data, and measure completed tasks, not minutes saved.
What conversational AI handles well in healthcare
The calls that work best are frequent, repetitive, and have a right answer that lives in a system the agent can query.
| Patient need | What the agent does | What it must connect to | When it hands off |
|---|---|---|---|
| Book or move an appointment | Finds open slots, books, confirms date, time, and location | Scheduling or practice management | No slots fit, or the visit type needs triage |
| Appointment reminders | Calls or texts, confirms or reschedules, logs the answer | Schedule plus a contact list | The patient asks a clinical question |
| Refill requests | Takes the medication name, pharmacy, and prescriber | A request queue for the care team | Always, for the clinical decision |
| Office information | Hours, directions, parking, what to bring | An approved knowledge source | The answer isn’t in the source |
| Intake before a visit | Collects demographics, insurance, and reason for visit | Intake forms or the EHR | The patient can’t or won’t answer |
| Billing and insurance | Explains a statement, takes a payment, checks accepted plans | Billing system, payer list | Disputes and coverage decisions |
| Payer calls (provider side) | Calls insurers to verify benefits or check a claim | Claims data, a phone line out | The payer rep needs a human |
The last row is easy to forget, because no patient is involved. A large share of a practice’s phone time is staff on hold with insurers. SuperDial builds agents that make those calls, navigate payer phone trees, and wait on hold, and reports billing teams getting four times as much done.
Chat or voice?
Chatbots on a patient portal answer people who already went to the portal. The patients who call are often the ones who didn’t: older patients, patients with a question too messy for a form, patients who want to reschedule from the car. If the goal is to answer more of the demand you already have, the phone line is where it is.
Voice is also harder. A phone agent has to understand speech over 8 kHz phone audio, decide when the caller has finished talking, and reply fast enough that the silence doesn’t feel like a dropped call. Names, dates of birth, and drug names have to come through correctly in both directions. A chatbot that misreads “Lisinopril” shows a typo; a voice agent that mishears it schedules the wrong refill.
Where the clinical line goes
The boundary that matters most is between administrative and clinical. An agent can tell a patient the clinic opens at 8. It should not interpret symptoms, adjust medication, or decide whether something is urgent, unless you’re building a clinical product with clinical validation and oversight.
Write the boundary into the agent’s instructions in plain terms: which tasks it handles, which topics it never answers, and what it says instead. Then test it the way a patient would. “I’m booking for my chest pain, can you just get me in Friday?” is a scheduling request and an emergency at the same time. The right answer is to direct the caller to emergency services or transfer them to a nurse line right away, not to offer Friday at 2.
Assort Health, which runs specialty-specific agents across more than 140 clinics, built its product around this: the agents handle the logistics calls and route the rest to staff, which frees people for the calls that need judgment.
HIPAA and conversational AI
HIPAA compliance is a property of the whole system, not something you buy with a model. For a voice agent, check five things:
- A business associate agreement with every vendor that sees patient data. That includes the speech recognition and speech synthesis provider, the language model provider, the telephony carrier if it records, and the platform that runs the agent. HHS publishes sample business associate agreement provisions if you want to see what one covers.
- Minimum necessary access. The agent should look up the one appointment it needs, not have read access to the whole chart. Scope each API the agent can call.
- Retention. Decide how long call recordings, transcripts, and logs are kept, and where. Some vendors offer zero data retention, which means they don’t store the audio and text they process.
- Identity checks. Before the agent reads back anything about a patient, it should verify who is calling, the same way your front desk does.
- Where it runs. Some health systems require a private cloud or on-premises deployment. Ask early, because not every vendor can do it.
Cartesia is HIPAA compliant and SOC 2 Type II certified, signs business associate agreements with healthcare customers, offers zero data retention for enterprise text to speech and speech to text, and deploys in your cloud or on premises. Details are in the Trust Center. Your privacy and compliance owners still need to review the full data path, because the agent touches your systems too.
Outbound calls: reminders and consent
Reminder and recall calls are some of the easiest wins, because the patient already has an appointment and the call has one job. They also carry a legal question: in the US, the FCC ruled in 2024 that AI-generated voices count as “artificial” voices under the Telephone Consumer Protection Act. Calls that use them follow the same consent rules as prerecorded calls. Confirm with counsel which of your call types need prior consent, and have the agent say up front that it’s an automated assistant calling for your practice.
What makes a patient trust the voice
Patients hang up on agents that sound wrong, pause too long, or mangle their name. Three details matter more in healthcare than elsewhere:
- Speed. Long gaps before a reply make callers think the line dropped, and they start talking over the agent. On Assort Health’s calls, Cartesia’s speech model returns the first audio in about 90 milliseconds. Assort reports 89% shorter wait times and 7% more booked appointments since deploying its agents.
- Hard words, both ways. Drug names, provider names, and street names need to be heard correctly and spoken correctly. On Cartesia, keyterms bias speech recognition toward the words you list, and a pronunciation dictionary fixes how the agent says them.
- Read-backs. Dates, times, and dates of birth should be confirmed out loud in a form people can check: “Tuesday, October 14th, at 9:30 in the morning, at the Dedham office.”
How to build a healthcare voice agent on Cartesia
On Cartesia Managed Agents, you configure the agent and Cartesia runs the speech recognition, language model, speech synthesis, and phone connection.
- Write the instructions. Define the role, the tasks the agent handles, the topics it never answers, and the exact transfer behavior for anything clinical or urgent.
- Connect your systems with webhook tools. A webhook tool lets the agent call your scheduling or practice management API, for example
check_availabilityandbook_appointment. Keep each one narrow. - Add a transfer to staff. The
transfer_to_numbersystem tool sends the call to a phone number or SIP address when a condition you write is met, such as “The caller describes symptoms or asks a medical question.” See the warm transfer guide for passing context to the person who picks up. - Teach it your vocabulary. Add provider names, drug names, and location names as
keyterms, and add a pronunciation dictionary for any it says wrong. - Personalize outbound calls. For reminders, pass the patient’s first name and appointment time as dynamic variables in the outbound call request, so the agent doesn’t need record access to open the call.
{
"config": {
"instructions": "You are the scheduling assistant for Riverside Orthopedics. Help callers book, move, or cancel appointments and answer questions about hours and locations. Verify the caller's name and date of birth before discussing any appointment. Never give medical advice. If the caller describes symptoms, asks about medication, or sounds distressed, transfer them to the nurse line.",
"initial_message": "Thanks for calling Riverside Orthopedics. This is the automated scheduling assistant. How can I help?",
"audio": {
"input": {
"keyterms": [
"Riverside Orthopedics",
"Dr. Okafor",
"Dedham",
"meloxicam"
]
},
"output": { "voice_id": "e07c00bc-4134-4eae-9ea4-1a55fb45746b" }
},
"system_tools": {
"transfer_to_number": {
"pre_tool_speech": "force",
"transfers": [
{
"destination": { "type": "phone", "phone_number": "+16175550142" },
"condition": "The caller describes symptoms, asks a medical or medication question, or asks for a nurse."
},
{
"destination": { "type": "phone", "phone_number": "+16175550100" },
"condition": "The caller asks for a person at the front desk."
}
]
}
}
}
}How to pilot it
Pick one call type at one location, put the agent on that line during business hours with staff as the fallback, and review calls every day for the first two weeks. Track:
- Completed tasks. Appointments booked or moved without a human, as a share of the calls that asked for that.
- Transfers by reason. Clinical transfers should happen every time they’re needed; avoidable transfers are your backlog.
- Corrections. Bookings staff had to fix, and why.
- Hang-ups before resolution. Most of these are latency or comprehension problems.
- Boundary tests. A weekly set of scripted calls that try to get medical advice out of the agent. It should fail every one.
When the numbers hold for a few weeks, extend hours first (nights and weekends are where unanswered calls pile up), then add the next call type.
To try it, create an agent in the Cartesia playground, connect a test scheduling endpoint, and call it from a real phone. The healthcare page has a live demo of a scheduling agent.