A change to an agent can pass a prompt test and still break the workflow around it. A new instruction may select a different tool. A revised router may omit an argument that used to be present. A recovery path may now retry after a conflict instead of re-reading the record. If the only test asks whether the final prose sounds right, the broken decision can remain hidden until the agent reaches a live system.
The practical move is to replay representative tool traces before releasing an agent change. Save a small set of completed, scrubbed runs as structured inputs and tool receipts. Run the candidate against a controlled tool double that returns the recorded—or deliberately varied—receipts. Grade the whole decision path: selected tool, validated arguments, guardrail outcome, stop reason, and final claim. The candidate earns live access only after it behaves correctly against the cases that matter.
OpenAI describes a trace as the end-to-end record of model calls, tool calls, guardrails, and handoffs, and recommends trace grading for diagnosing workflow-level failures.[1] Its trace-grading guidance makes the distinction explicit: the purpose is to label decisions and tool calls as well as the final output, so regressions can be found and understood rather than treated as black-box failures.[2] This is the useful test boundary for agents: evaluate the route that produced the answer, not only the answer itself.
Record a scenario, not a transcript
A replay fixture should preserve the facts the runtime would have seen, while excluding secrets and irrelevant chat history. Start with runs that represent a real decision boundary: a normal lookup, a missing identifier, an authorization denial, a stale-record conflict, a transient timeout, and an ambiguous request that must stop for review.
For each fixture, save the incoming task, authorized scope, available tool surface, and ordered tool receipts. Include stable fake IDs and versions, not live customer data or credentials. A tool double should match the real tool’s name and input/output contract closely enough that the agent cannot pass by relying on a test-only shortcut. Temporal makes the same testing point for workflow code: a mocked activity should keep the real activity’s name, signature, and input/output types, so the workflow exercises the same interface while its effect is isolated.[3]
Do not call a capture from one lucky run a test suite. A useful collection includes the paths where the agent must refuse a tempting next step. The point is not to reproduce every thought token. It is to reproduce the externally meaningful choices and evidence boundaries.
Copy this tool-trace replay card
Use one card for a fixture or a small scenario family. Keep it beside the agent’s versioned prompt, routing policy, and tool contracts.
SCENARIO
- scenario_id: [stable ID]
- purpose: [decision or failure boundary under test]
- input: [scrubbed user task]
- authorized_scope: [tenant, records, tools, and effect limits]
- available_tool_surface: [exact named tools + schema version]
CONTROLLED TOOL RECEIPTS (in expected order)
1. tool: [name]
accepted_arguments: [validated shape or predicate]
returns: [recorded success | conflict | timeout | denial receipt]
2. tool: [name]
returns: [...]
MUST OBSERVE
- allowed tool sequence: [named order or permitted branches]
- argument checks: [target ID, version, field allowlist, operation ID]
- must stop as: [completed | needs_review | conflict | blocked]
- must not: [invoke a write | broaden scope | retry with a new ID | claim success]
- final evidence: [receipt IDs or cited facts required]
RESULT
{scenario_id, candidate_version, passed_checks, failed_check,
trace_ref, final_outcome, release_decision}
A timeout fixture is especially revealing. If the fake send tool returns an unknown outcome after accepting an operation ID, the candidate should look up that same operation—not issue a fresh send. If a fake update returns a version conflict, it should stop or re-read according to policy—not force the write. Those are observable checks that a final-answer preference grader would miss.
Replay the good path and the nearly-good path
Begin with traces taken from completed runs, then derive narrow variants. Keep the same request but change one controlled receipt: a selected record has a new version; a tool returns needs_approval; the catalog no longer exposes the previously selected tool; a paginated list says it is incomplete. This turns past production behavior into nearby boundary tests without inventing a large synthetic dataset.
Run the baseline and candidate against exactly the same fixture set. Compare structured receipts, not just a pass percentage. A candidate that improves routine answers but starts a write after an approval denial has a release-blocking regression, even if its average score rises. OpenAI recommends moving from individual traces to repeatable datasets and eval runs once good behavior is understood; that makes a fixed fixture set a practical release gate rather than an occasional debugging exercise.[1]
Keep the test executor in control. The model can choose among the tools you expose, but the fixture harness decides what each call returns and rejects calls outside the approved surface. Never connect a replay test to a production sender, publisher, payment provider, or mutable customer record merely because the test input was scrubbed.
Failure mode: replaying yesterday’s happy path
A replay suite can become a museum. It may encode an old tool schema, an obsolete policy, or only the successful path that first inspired it. Passing it then proves compatibility with history, not safety for today’s workflow. A deterministic replay can also prove that orchestration code remains compatible with a recorded history while saying nothing about whether a new model’s judgment is appropriate.
Refresh fixtures after real incidents, near misses, policy changes, and tool-contract changes. Keep a small human-reviewed set for ambiguous judgment calls, and pair replay with limited shadow traffic when a change is substantial. The goal is not to simulate production perfectly. It is to turn known consequential paths into repeatable evidence that an agent change still selects, calls, stops, and reports safely.
The habit to keep
before an agent change gets a live tool, make it walk the recorded path first. Replay the receipts that define safe behavior, grade the decisions along the way, and block releases that reach the right sentence by taking the wrong route.