Definition
A knowledge base is a collection of documents, such as FAQs, policies and product pages, that a voice agent can search while answering a caller. It gives the LLM facts to draw on that were not in its training or its instructions.
How a knowledge base works
An LLM knows only what it was trained on plus what is in its instructions. Your return policy, this month’s prices and the steps to reset a router are in neither. A knowledge base holds that material so the agent can look it up.
The usual pipeline has two parts.
- Preparation. Documents are split into passages, and each passage is indexed so it can be found later. Many systems index by meaning, using numeric embeddings, which means a question can match a passage that uses different words.
- At answer time. The caller’s question, or a rewritten form of it, is used to search the index. The best-matching passages are added to the LLM’s input, and the model answers from them. This pattern is described under retrieval-augmented generation.
In Cartesia’s Managed Agents, knowledge base content is organized into folders, and folders are attached to an agent.
A knowledge base is for facts that change or that are too long to place in the instructions. Short rules that apply to every call belong in the system prompt. Live data for one customer, such as an order status, is better fetched with a tool from the system of record, since a document store is only as current as its last update.
Why it matters for voice agents
A knowledge base reduces invented answers, but only when it works. Three problems show up on calls that rarely show up in text.
First, the search runs while the caller waits, so it adds to voice latency. Second, the question arrives as a transcript, and a misheard word can make the search miss the right passage. Third, passages written for reading are long, and read aloud they run on. Rewrite the source into short, speakable answers, or tell the model to summarize in a sentence or two.
Content quality decides the outcome. Old, contradictory or duplicated documents produce confident wrong answers. Assign an owner, date the pages, and remove what is out of date. Decide what the agent says when nothing relevant is found: “I don’t have that, I can transfer you” beats a guess. See hallucination.
Example
An illustrative exchange:
- Caller: “Can I return something after thirty days?”
- The agent’s search finds the returns page: “Returns are accepted within 45 days with a receipt.”
- Agent: “Yes. We accept returns within 45 days if you have the receipt.”