A prompt can look excellent after three friendly examples and still fail in the first real workflow. That is not because the examples were useless; they were just too similar to the cases you expected. The useful move before connecting a model to a queue, database, or customer-facing action is to make a tiny breakpoint set: a handful of inputs chosen to expose the decisions that matter when the request is vague, conflicting, unusual, or unsafe to guess.
This is a practical evaluation habit, not a research project. OpenAI describes evaluations as tests that check whether model outputs meet specified style and content criteria, and frames the loop as describing the task, running representative inputs, then analyzing and improving the result.[1] Anthropic likewise recommends success criteria that are specific and measurable, with evaluation cases that mirror the real task distribution and include edge cases.[2] The missing step in many everyday AI workflows is choosing test inputs deliberately before a prompt change gets trusted.
Find the decision, then find its breaking point
Start with one action the output will drive. Maybe an assistant labels a support request, extracts an invoice field, or drafts a response that a teammate sends. Ask: what would make this output costly, embarrassing, or time-consuming to repair? Write the answer as a checkable rule.
For a support triage workflow, “be helpful” is not a rule. These are:
- Route a billing complaint to billing, not general support.
- Do not call an outage urgent unless the message supplies evidence.
- Ask for clarification rather than invent an account detail.
- Preserve a stated deadline in the handoff.
Then create inputs that pressure each rule. A useful first set has four buckets:
- Routine: a clear, ordinary example that should pass.
- Boundary: an input just on either side of a policy or category line.
- Conflict: an input with two plausible interpretations or contradictory details.
- Unknown: an input missing the information needed for a safe answer.
The point is not to catch every conceivable failure. It is to prevent a prompt from being approved only because it handled easy cases. Keep the examples recognizably close to the work your team actually receives. Anthropic specifically advises task-specific evaluation cases rather than generic tests.[2]
Copy this breakpoint-set template
Put this in a document, spreadsheet, or test fixture beside the prompt. Each row has a single expected behavior, so a changed result has somewhere concrete to land.
WORKFLOW: [name the action the AI output enables]
DECISION UNDER TEST: [the choice that can go wrong]
CASE ID: [short name]
BUCKET: routine | boundary | conflict | unknown
INPUT: [realistic input, scrubbed of sensitive data]
EXPECTED BEHAVIOR:
- Must: [observable requirement]
- Must not: [prohibited behavior]
- If uncertain: [clarify, route for review, or return a named fallback]
CHECK:
pass | fail | needs human review
NOTES: [quote the output or explain the mismatch]
For a four-case support set, you might use a normal refund request, a request that mentions both a duplicate charge and a login problem, a message saying “the app is down” with no scope, and a compliment with no request. The desired answer is not necessarily a perfect diagnosis. It is a reliable next move: route, preserve ambiguity, or ask a person for the missing fact.
Grade the cheapest reliable thing first
Not every check needs another model. If a response must include a ticket ID, use a simple presence check. If it must choose one approved category, compare the value directly. OpenAI’s evaluation example pairs labeled test data with a string-equality criterion; Anthropic recommends choosing the fastest, most reliable, scalable grading method and identifies code-based checks and exact match as strong options for rule-like requirements.[1][2]
Reserve human review for judgments that cannot honestly be reduced to a rule: whether the explanation is fair, whether a proposed reply would damage trust, or whether evidence supports a nuanced conclusion. If you use an LLM to grade, give it a narrow rubric and sample its judgments against a person’s review before trusting its score. A fluent grader can repeat the same blind spot as the system it evaluates.
Change one variable, compare every case
When revising a prompt, model, retrieval source, or tool instruction, rerun the whole breakpoint set. Do not look only at the case you were trying to fix. Record which cases changed and why. A revision that improves the refund example but starts guessing on incomplete requests is a regression, not a win.
Small sets are valuable because they can be rerun often. Add a case only after a real miss, a near miss, or a new class of work appears. Over time, the set becomes a compact memory of the ways your workflow has already surprised you.
Failure mode: treating the score as the goal
A breakpoint set can create false confidence if it becomes frozen, unrepresentative, or too easy to game. Four passing examples do not prove that an automation is safe for every user. They only show that it met four declared expectations. Refresh examples from real work, protect personal information, and keep an explicit human-review path for consequential decisions. The test is a guardrail for change—not permission to stop watching the road.
The habit to keep
before you celebrate a better-looking AI response, name the decision it affects and write the few inputs most likely to expose a bad call. Test those every time you change the system.