1. Check the Overview Dashboard Daily
The Overview dashboard is your command center. Make it a habit to check it daily to monitor your agent’s health.
| Metric to Watch | What It Tells You |
|---|---|
| Success Rate | A sudden drop is a clear sign that something is wrong. This could be due to a bug, a change in user behavior, or an issue with your agent’s logic. |
| Failed Calls | Refers to calls with the outcome defined as “Failure” in your Agent Configuration. A high or increasing number indicates systemic issues that need investigation. |
| Hallucination Failures | Identifies when your agent is providing incorrect information. This is a critical metric to watch as it can directly impact user trust. |
| Missed Intent Rate | Suggests your agent is struggling to understand users. This could be due to a lack of training data, a poorly defined intent, or a change in how users are interacting with your agent. |
2. Investigate Red Flags Immediately
The Recent Red Flags section is your early warning system. You can find it at the bottom of the Overview dashboard. Prioritize and investigate these as soon as they appear.
- Prioritize by Flag Type: Focus on critical failures like “Agent Hallucination” or “Broken Flow.”
- Drill Down: Click on a red-flagged call to understand the context of the failure.
3. Use Call Logs to Spot Patterns
Use the Call Logs to uncover deeper patterns. You can access the Call Logs by clicking on Call Logs in the sidebar.- Filter by Outcome: Regularly filter by “Failure” to review recent unsuccessful interactions.
- Search for Keywords: Find calls that mention relevant keywords to investigate specific issues.
4. Configure Meaningful Alerts
Set up Alerts in Agent Settings to be notified of the events that matter most. A well-configured alert can be the difference between catching a problem early and having it affect a large number of users. Here are some examples of meaningful alerts you can create:- Critical Red Flag Spike: Get an email notification if you get more than 5 Hallucination red flags in a 1-hour window. This can help you quickly identify if your agent is providing incorrect information.
- Poor Customer Experience: Create a “Warning” level alert if the average Politeness / Tone score drops below 3 for more than 10 calls in 15 minutes.
- Agent Stuck in Loop: Get a critical alert if the Agent stuck in loop red flag appears more than 3 times in 5 minutes.
5. Establish a Weekly Review Cadence
Set aside time each week for a more thorough review.- Review Long-Term Trends: Use the 30-day and 90-day filters to understand performance over time.
- Analyze User Intent Breakdown: Look for trends in what users are asking for.
6. Validate Changes with Data
Whenever you make a change to your agent, use Tuner to validate its impact.- Compare Before and After: Use the date filters to compare performance before and after a change.
- Look for Unintended Consequences: A change that improves one metric may negatively impact another.
Next Steps
Now that you have a framework for monitoring, dive deeper into the concepts that power Tuner.Key Concepts Overview
An overview of the core concepts that power the Tuner platform.