Best practices for agent monitoring

​

Check the Overview dashboard daily

The Overview dashboard shows your agent’s health for the period you pick. Check it daily.

​

Configure meaningful alerts

Set up alerts for the events that matter most, so you hear about a problem early. Examples:

• Red flag spike: a Critical alert when more than 5 calls in a 1-hour window have a red flag such as hallucination_detected.
• Eval failures: a Warning alert when the share of calls that fail your politeness eval passes a percentage you choose.
• Agent stuck in a loop: a Critical alert when a custom agent_stuck_in_loop red flag appears on more than 3 calls in a short window.

To create an alert, go to Agent Settings > Alerts and click Create Alert. Set the conditions, the threshold, the severity and the recipients.

​

Investigate red flags quickly

The Recent Red Flags section at the bottom of the Overview dashboard lists the latest red-flagged calls. Review them as soon as they appear.

• Prioritize by flag type: start with critical failures like hallucination_detected or compliance_failure.
• Open the call: click a red-flagged call to see the eval results behind the flag. For example, if a flag built on the Tool call missing eval is raised, the eval explains which tool call was missing.
• Target your next simulation: when the same red flag keeps appearing, note which eval triggered it and narrow Pressure Tests to it in your next run.

​

Use Call Logs to spot patterns

Open Call Logs from the sidebar to look across many calls.

• Filter by outcome: regularly filter to failure outcomes to review recent unsuccessful calls.
• Search: find calls that mention a keyword you’re investigating.

​

Review weekly

Set aside time each week for a longer review.

• Review long-term trends: switch the Overview period to 30 days or 90 days.
• Check the Call Intent Breakdown: look for changes in what callers are asking for.

​

Diagnose before you guess

When the dashboard shows a problem (a drop in success rate, a spike in one red flag, an intent that keeps failing), you could review calls one by one. Instead, click Diagnose Agent on the Overview page and run the diagnosis through the MCP server. The assistant reads your dashboards, red flags, call logs and transcripts, finds the pattern, and reports the root cause and a fix. It also finds calls that don’t match any of your configured intents.

​

Validate changes with data and simulation

After you change your agent, check the impact before more live traffic reaches it.

• Compare before and after: use the Overview period to compare production performance before and after the change.
• Run Call Simulation: after updating a prompt, workflow or integration, run a simulation to confirm the fix and check for regressions. Simulation uses the same evals as production, and simulated calls are excluded from Overview metrics.
• Replay real calls: replay the calls in your Dataset against the updated agent. The caller audio is identical on every run, so any change in the results comes from your update.
• Look for side effects: a change that improves one metric can hurt another. A simulation run shows this sooner than waiting for enough production calls.

​

Next steps