Darkmind / Issue 014

Review the Diff, Not the Story

Give an AI a narrow change budget, then judge the patch and its checks—not the confidence of its explanation.

Oct 21, 20264 min readReliable coding

An AI coding assistant can describe a change and still touch the wrong file, remove a guard that looked unrelated, or solve one symptom by adding a dependency nobody intended to ship. The deliverable is the concrete delta between the repository you trusted and the repository you are about to accept.

The practical technique is to give the agent a change budget, then review the resulting diff against that budget before you accept the work. A change budget is a small, testable contract: the named files it may touch, the behavior that must change, the behavior that must not change, and the checks that must pass. It turns “fix the login bug” into a reviewable boundary rather than an invitation to rewrite nearby code.

OpenAI’s Codex guidance recommends supplying a goal, relevant context, constraints, and a definition of done; it also says to ask the agent to run relevant checks and review the work before accepting it.[1] Git’s git diff documentation defines the tool as showing changes between the working tree, index, commits, or files, which makes it a practical independent view of the patch rather than the agent’s account of it.[2] Together, they suggest a useful division: state the intended change before implementation, then evaluate the actual patch afterward.

Give the edit a budget before it starts

A useful budget is narrow enough that a reviewer can reject a change for crossing it. Name the target behavior and the allowed surface. For example, “accept a missing optional nickname field in the profile endpoint” is not the same task as “modernize profile validation.” The first might authorize a handler, its validation test, and an API fixture. It does not automatically authorize a database migration, a new package, or a UI rewrite.

Copy this change-budget prompt

Use this before an AI edits a repository. Replace the brackets with details that a reviewer can later check in the diff.

TASK
Make this specific behavior change: [observable outcome].

ALLOWED CHANGE SURFACE
- Files or directories allowed: [explicit paths]
- Files that may be added: [none, or named paths only]
- Dependencies/configuration changes: [none, or explicit list]
- Public/API behavior allowed to change: [one named behavior]

MUST PRESERVE
- [existing behavior, compatibility rule, security boundary, or performance limit]
- [tests, interface, or files that must not change]

DONE WHEN
- [named test/check] passes
- [manual observable check] produces [expected result]
- `git diff --check` reports no whitespace errors
- The final diff stays inside the allowed surface, or every exception is listed
  under SCOPE CHANGE NEEDED.

FINAL RECEIPT
1. Changed files: [path + one-line reason]
2. Behavior changed: [testable statement]
3. Checks run: [command + result]
4. Scope change needed: [none, or exact extra file/reason]
5. Review cues: [risky hunk, migration, compatibility note, or none]

DO NOT
- Refactor adjacent code merely for style.
- Add packages, change lockfiles, or alter CI/configuration unless authorized.
- Claim a check passed unless you ran it and report its result.

The MUST PRESERVE section is where many vague coding requests become safe to review. If callers depend on a response field, name it. If authentication must stay server-side, say so. If a file is generated, declare whether the source, generated output, or both are in scope. Good agent context reduces assumptions; the budget gives those assumptions a place to be challenged before they turn into edits.[1]

Review in two passes

First, scan the change shape before reading every line. Use git diff --stat or git diff --name-status to answer: which files changed, were files added or deleted, and is the change proportionate to the request? A one-line bug fix that modifies a lockfile, an environment template, and six unrelated modules deserves a pause even if the tests are green. Git documents options for file names, status, and summary alongside patch output, so this first pass can stay deliberately high level.[2]

Second, inspect the patch hunk by hunk. For each modified block, ask four plain questions: Does it contribute to the promised behavior? Does it preserve the named boundaries? Does it introduce an untested branch or a new external effect? Is the test proving the outcome, rather than merely exercising the new code? Compare the final receipt to the diff; the diff wins when they disagree.

Keep verification separate from review. A passing unit test supports a narrow behavior claim, but it does not prove the patch avoided unrelated changes. Conversely, a clean small diff does not prove the changed behavior works. Run the named checks, inspect their real result, and then review the actual changed lines. Those are complementary checks, not substitutes.

Failure mode: turning a budget into a blindfold

An over-tight budget can block a legitimate fix. If the agent discovers that the correct repair needs a shared validator, a migration, or a compatibility update, do not force it to hide the work in an allowed file. Stop and make the new scope explicit. The budget controls surprise; it should not reward superficial patches.

Also avoid treating a small diff as inherently safe. A one-line change to authorization, pricing, or deletion logic can have more impact than a large documentation patch. Match the review depth and test plan to the consequence of the behavior, not just the number of changed lines.

The habit to keep

ask AI to work within a visible budget, then make acceptance depend on the patch, the checks, and any declared scope expansion. A confident story can help you navigate the change. The diff is what you are actually approving.

Sources

  1. OpenAI, “Codex best practices”
  2. Git, “git-diff Documentation”