| Term | Definition |
|---|---|
| Agent | The voice AI application that you are monitoring with Tuner. An agent is typically a conversational AI that handles phone calls, such as a customer support bot, appointment scheduler, or sales assistant. In Tuner, you configure each agent separately to define its evaluation criteria and monitoring settings. |
| Alert | A proactive notification sent when a specific event occurs, like a spike in red flags or a drop in success rate. Alerts can be delivered via email or webhook (which can route into tools like Slack), helping you catch issues before they impact more users. See Introduction to Real-Time Alerts. |
| Eval | A custom evaluation criterion that verifies whether your agent performed a specific action or followed certain guidelines during a call. For example, you might create an eval to ensure the agent always confirms appointment details before ending the call. See Introduction to Evaluations & Guardrails. |
| Call Logs | The page where you can view a list of all your calls, filter by various criteria, and search for specific conversations. Call Logs is your primary tool for investigating individual calls and spotting patterns across multiple interactions. See How to Analyze Your First Call. |
| Call Outcome | The result of a call, typically categorized as “Success,” “Failure,” or other custom outcomes you define. You can add common outcomes from Tuner’s predefined outcome library or create your own. Call outcomes help you quickly understand whether the agent achieved its goal and are used to calculate your overall success rate. |
| Call Simulation | A feature that runs real SIP test calls with your agent using AI simulation agents, dialing in for inbound agents, or having your agent dial out for outbound agents. Each simulated call is scored against the same Evaluations, Intents, and Call Outcomes configured for production. See Introduction to Call Simulation. |
| Evaluation | The process of analyzing a call against a set of criteria to determine its quality. Evaluations run automatically on every call, checking for specific behaviors, measuring voice metrics, and applying guardrails. Think of it as an automated quality assurance process. See Introduction to Evaluations & Guardrails. |
| Guardrail | Rules that define what “bad” looks like for your agent. Guardrails act as safety nets. When a call breaches a guardrail (e.g., the agent hallucinates or misses an intent), it automatically triggers a red flag or alert. See Introduction to Evaluations & Guardrails. |
| Hallucination | When the agent provides incorrect, fabricated, or misleading information that wasn’t part of its training data or knowledge base. Hallucinations are critical issues because they can erode user trust and lead to real-world consequences, such as booking the wrong appointment or providing incorrect pricing. |
| Intent | The user’s goal or purpose in the conversation. For example, in an appointment scheduling agent, common intents might include “Book Appointment,” “Reschedule,” “Cancel,” or “Check Availability.” Understanding intent distribution helps you optimize your agent for the most common use cases. |
| Metric | A quantitative measurement computed for every call, such as talk time, silence duration, talk-to-listen ratio, or sentiment score. Metrics provide objective data about conversation quality without requiring manual review. See Introduction to Pre-defined Metrics. |
| Overview | The main dashboard showing high-level performance metrics for your agent, including total calls, success rate, failed calls, hallucination failures, and missed intent rate. The Overview is your command center for monitoring agent health at a glance. See Best Practices for Agent Monitoring. |
| Pressure Test | A simulation agent persona designed to stress-test your agent with difficult scenarios, such as unhappy customers, edge cases, scope-boundary requests, or attempts to bypass your process. Pressure Tests are auto-generated from your configured evals. See Introduction to Call Simulation. |
| Red Flag | A tag applied to a call when a rule you define matches — for example an eval failure like a hallucination, a negative sentiment, or a failed outcome. Red flags appear in the Overview dashboard and Call Logs, making it easy to prioritize which calls need immediate attention. See Configure Red Flags & Labels. |
| Routine Caller | A simulation agent persona that follows your agent’s normal, cooperative workflows. Routine Callers are generated from your configured Intents and Call Outcomes to test the happy path. See Introduction to Call Simulation. |
| Rule Builder | The interface for creating custom rules that determine when labels and red flags are applied to calls. The Rule Builder allows you to combine multiple conditions (evals, metrics, metadata) using AND/OR logic to create sophisticated detection rules. |
| Scenario | The conversation script Tuner generates for each simulated call, including the simulation agent’s persona, intent, and what to stress-test. Scenarios are built automatically when you run a simulation batch. |
| Sentiment | A measure of the emotional tone in the conversation, typically categorized as positive, negative, or neutral. Sentiment analysis helps you understand how users feel during interactions with your agent, which can indicate satisfaction or frustration. |
| Simulation Batch | A group of simulated calls run together in one session. Each batch runs 5–20 calls, configured in the Run Simulation dialog. |
| Simulation Mix | The ratio of Routine Callers to Pressure Tests in a simulation batch. You adjust the mix with a slider in the Run Simulation dialog, from all routine to all pressure or anywhere in between. |
| Simulations Table | The page in Tuner where completed simulation calls are listed with type, intent, outcome, tags, sentiment, ended reason, duration, and cost. Click any row for the full transcript, latency breakdown, tool traces, and evaluation results. |
| SIP | Session Initiation Protocol, the signaling standard voice platforms use to start, manage, and end calls. Tuner uses SIP so simulation calls travel the same path as a real phone call. See Simulation SIP setup. |
| SIP Call-ID | A unique identifier carried with each SIP call. In inbound simulation, Tuner uses it to match the call it dialled with the session your agent syncs back; custom integrations pass this as sip_call_id in the Create Call API payload. Outbound simulation links calls via the recipient field instead. |
| SIP Settings | The Agent Settings → SIP Settings configuration where you paste the SIP URI and credentials Tuner uses to place inbound simulation calls. Outbound agents are configured in Agent Settings → Simulation Setup instead. |
| SIP URI | A SIP address, the internet equivalent of a phone number, for example sip:agent@sip.vapi.ai. In inbound simulation, Tuner dials your agent’s SIP URI; in outbound simulation, Tuner gives your system a SIP URI to dial. See Simulation SIP setup and Outbound Simulation. |
| Transcript | A written record of the conversation between the user and the agent. Transcripts include timestamps and speaker labels, allowing you to review exactly what was said and when. They are essential for root cause analysis and debugging issues. |
Next Steps
Now that you’re familiar with the terminology, dive into the detailed guides for each concept.Introduction to Evaluations & Guardrails
Learn how evaluations and guardrails work to improve your agent’s quality.
Introduction to Call Simulation
Learn how to run simulation batches and stress-test your agent.