Evaluation is configured per agent in Agent Settings: outcomes, intents and data extraction under Analysis Setup, and evals and rules under Evaluation Rules.
What is evaluation?
Evaluation checks every call against criteria you define, so you don’t have to listen to calls one by one. Tuner tells you whether a call failed and why, by checking specific behaviors, metrics and outcomes. Guardrails define what “bad” looks like for your agent. In Tuner they are rules that tag matching calls with a red flag, and alerts that notify you when matching calls cross a threshold.How it works
1
Evals score the call
Tuner runs your evals on every call and computes voice metrics.
2
Rules tag the call
Each label and red flag rule checks its conditions against those results and tags the call when they match.
3
Alerts notify you
Alerts watch calls over a time window and notify you by email or webhook when a condition crosses your threshold.
The same evals and rules run on every production call and every Call Simulation call, including replays of the real calls in your Dataset. What you configure here powers both monitoring and testing.
Types of evaluation
- Evals
- Voice Metrics
- Labels & Red Flags
Evals check whether your agent performed specific actions or followed specific guidelines during a call. Tuner runs them on every call.Predefined evals. Tuner’s library has 27 evals across Accuracy, Actions & Tools, Conversation Quality and Safety & Compliance. New agents start with Hallucination, Prompt injection resistance and Politeness.Custom evals. Define your own criteria in plain language. Tuner runs them alongside the predefined ones.Example: use the Confirmation accuracy eval to verify that the agent’s final confirmation matched the action it completed, or create your own “Appointment Confirmed” eval for your workflow.See Predefined evals or Create evals.
What an eval judges
Every eval is run by an LLM judge, and that judge sees more than the transcript. What you put in front of it is what it can score against.
So an eval can check facts as well as tone. “Was the agent polite” is a judgment call. “The agent quoted a delivery time the tool never returned” is a fact check against the tool output. Both run as evals on the same call.
Evals and languages
Each judge reads the transcript in the language the call happened in, so an agent taking Arabic, French or Spanish calls is scored by the same evals as an English one. There is no per-language setup, and a multilingual agent needs only one set of evals. You can write eval prompts in English or in the language of the call.Language support in Call Simulation is a separate list, because Tuner has to speak on those calls as well as read them. See Languages and accents.
Common use cases
Detect hallucinations
Add the Hallucination eval and a red flag rule that tags calls where it fails.
Ensure compliance
Create an eval that checks the agent reads a required disclaimer.
Monitor sentiment
Create an alert that notifies you when caller sentiment is negative on too many calls.
Next steps
Predefined evals
The 27 evals in the library and what each one checks.
Red flags and labels
Turn eval results and metrics into tags on your calls.