Definition
A large language model (LLM) is a neural network trained on huge amounts of text to predict the next token in a sequence. By repeating that step it writes text, follows instructions given in a prompt, and can decide to call tools.
How a large language model works
An LLM reads text as tokens, which are pieces of words, and outputs a probability for each possible next token. It picks one, appends it, and repeats until the reply is complete. This is why the first words of an answer can be heard well before the last ones exist, and why generation time grows with the length of the answer.
Most current models use the Transformer design introduced by Vaswani and colleagues in 2017, which relies on attention to relate each token to the others in its context. The context window is the limit on how much text the model can consider at once, covering instructions, conversation history and any documents or tool results added to it.
Brown and colleagues showed in 2020 that a model with 175 billion parameters could perform new tasks from instructions and a few examples written in the prompt, without retraining. That is the basis for building agents by writing instructions rather than training a model per task.
Two further limits matter. The model has no built-in access to facts after its training, so applications add information through the prompt or through tools. And it can produce fluent text that is false, a failure called hallucination.
Why it matters for voice agents
In a cascaded voice agent the LLM is the stage that decides what to say, and usually the slowest and least predictable one. The number that counts is time to first token, because speech can start as soon as the first sentence is ready if the output is streamed. Total generation time matters less than that.
Voice sets extra demands. Replies should be short and written for the ear, with no lists or markdown. The model must follow its system prompt under pressure from callers who change topic mid-sentence. And any claim it makes about a price, a policy or an account is spoken aloud with confidence, so grounding it in tools and records matters more than in a chat window where a reader can double-check.
A speech-to-speech model folds this stage into the audio model, so the LLM and the voice are no longer separate choices.