Skip to main content
Your Scenario Library lives on the Scenarios tab under Simulations. To rerun Scenarios, you need an agent set up for inbound or outbound simulation, and at least one Pass/Fail eval.

What is the Scenario Library?

Voice agents change all the time: a new prompt, a new model, a new voice. Any change can quietly break a call that used to work. The Scenario Library is how you check. It’s where your Scenarios live, and together they are your regression suite. A Scenario is one saved situation for a Simulated Caller to play on a real voice call with your agent. Write it once, rerun it after every change, and Tuner scores each call with the same Evals you already use. A Scenario that passed last release and fails today shows up as a Failure. A normal simulation writes new Scenarios every time, and they live only inside that run. In the library, a Scenario is saved on its own, so you can keep it and rerun it whenever you want.
The Scenarios tab with five saved Scenarios, showing each one's type, origin, tags, and last result

Two ways to add a Scenario

Write a Custom Scenario

Write the situation yourself, like a rushed patient or a customer who keeps pushing back. You don’t need to run anything first.

Save to library

Keep an AI-generated Scenario from a simulation you already ran, like one that caught a Failure.
Either way, you rerun them the same way.

Write a Custom Scenario

1

Open the Scenario editor

Go to Simulations > Scenarios and click New scenario.
The New scenario form with a name, type, situation, goal, stop conditions, intent, target eval, and tags filled in
2

Describe the Scenario

Only Name and Situation have to be filled in.
  • Type is Routine for an everyday request or Pressure for a difficult caller. It’s a label for scoring and filtering. Your Situation is what makes the Simulated Caller act that way, so describe any pushback there.
  • Situation is written to the caller, as “you”: who they are, their mood, what they want, and how they react when things don’t go their way. For example, “You’re calling from your car to move your cleaning to Friday the fourteenth. The line is noisy and you’re in a hurry.”
  • Goal, First line, and Stop conditions are optional. They say what the caller is trying to get done, the first thing they say, and when they’re done. A call still ends only when your agent hangs up or the max call duration is reached.
3

Choose how it's scored

  • Pick an Intent if the call is about one of your agent’s Intents. If your agent places calls, the Intent is also sent to your trigger endpoint as {{scenario.intent}}. See Outbound simulation.
  • For a Pressure Scenario, pick a Target eval, the Pass/Fail eval the call is meant to trip. The Scenario passes or fails on that eval alone.
  • Add Tags to group and find your Scenarios later.
4

Save

Click Save, or Save and run to save it and open Run Simulation with this Scenario selected. To check what the Simulated Caller is told from your text, expand Preview simulator prompt. It’s read-only and never blocks saving.
In the app, a Scenario you write yourself is labeled Manual.

Save to library from a run

1

Run a normal simulation

Start a run with AI generated Scenarios, as in Run a simulation, and wait until its calls have ended. You can save from a run that’s Analyzing, Complete, Partial, Unreported or Analysis failed. See Run status.
2

Pick the Scenarios to keep

On the Runs tab, expand the run and tick the calls you want to keep, one or several. A bookmark marks calls that are already in your library.
3

Save to library

Click Save to library, give each one a name, and optionally add Tags for all. Calls that didn’t connect can still be saved, since the Scenario itself is valid. Then click Save 2 scenarios (the number matches your selection). Your new Scenarios appear on the Scenarios tab, marked Saved from run.
A finished simulation run with two calls selected and the Save to library button
Tuner saves each call’s Situation, Goal, Stop conditions, type, Intent, and Target eval. It doesn’t save the language, background noise, or test profile, because you choose those each time you run. The call you saved from becomes the first entry in the Scenario’s Run history, so you can compare later runs against the Failure it caught.

Rerun after every change

A run started from your library is a Regression Run. Do one at launch, after every deployment, after a prompt edit, after swapping an LLM, STT, or TTS provider, and any time a production call shows you a new edge case.
1

Pick the Scenarios to rerun

Use whichever is closest:
  • On the Scenarios tab, open a row’s menu and click Run this scenario.
  • On a Scenario’s own page, click Run this scenario.
  • In Run Simulation, click Change next to Scenarios, choose From library, tick up to 20 Scenarios, and click Done.
The From library picker in Run Simulation with three Scenarios ticked
2

Set up the call and run

The Run Simulation dialog works as in any simulation. Choose the max call duration, language and accent, background noise, and test profile, then click Run. Each Scenario places one call, and you can rerun the same Scenario under different conditions each time. The dialog shows the number of calls and the credit rate before you run.

Read the results

The run appears on the Runs tab, marked with how many Scenarios came from your library. Every call is graded against all of your agent’s Pass/Fail evals, and a Pressure Scenario with a Target eval passes or fails on that eval.
  • On the Scenarios tab, Last result shows how each Scenario did on its latest run: Passed, Failed, Unreported, Analysis failed or Pending. A call that never connected also shows Failed. See Call results for what each one means.
  • Click a Scenario to open it. Run history shows how it did on its last runs, like “Passed 3 of last 5 runs”, with a View call link for each one. That’s how you see when it passed, when it hit a Failure, and that your fix held.
  • The Simulated Caller follows the situation but improvises, so the wording and length of each call can differ. To play back the exact same caller audio, use a replay of a Dataset call instead.
A saved Scenario with its situation and goal, and a run history of three passes out of the last five runs

Manage your Scenarios

  • Search by name, situation, goal, first line, or tag, and filter by Routine or Pressure.
  • Edit a Scenario at any time. Past runs keep the version they ran, so old results don’t change.
  • Duplicate a Scenario to start a new one from a copy.
  • Delete a Scenario to remove it from the library and from new runs. Past results are kept.

Limits

Scenarios are also available through the public API. Building and managing them is in the Scenarios group of the API Reference, and rerunning them (Run Saved Scenarios) or saving them from a run (Save Scenarios To Library) is in the Simulation group.

Next steps

Replays

Play back a real call from your Dataset with the exact same caller audio.

Introduction to Call Simulation

Generate new Scenarios to find failures you haven’t seen yet.