An AI editor says the bug is fixed.
That sentence is not evidence. Neither is a green test the editor wrote in the same pass.
Generated code can be excellent. It can also satisfy the visible example while weakening authorization, swallowing an error, changing an unrelated test, or moving the failure somewhere the prompt did not mention. The review needs to be independent of the explanation that came with the patch.
Here is the loop I use before merge.
Restate the failure without the proposed fix
Write one sentence describing the observable problem:
A signed-in member can request another team’s invoice by changing the invoice ID.
That is better than “the route forgot a tenant check” because it does not assume the implementation. Add the expected behavior and one negative case.
If the bug came from a scan or production report, preserve the original evidence. Do not let a long AI conversation replace the thing that actually failed.
Read the diff before the summary
Collapse the editor’s explanation and inspect the patch.
Start with the shape:
- Which files changed?
- Did configuration, dependencies, migrations, or generated files move too?
- Did the patch touch more of the application than the failure requires?
- Were tests weakened, deleted, skipped, or rewritten?
- Did error handling become broader?
A 12-file fix for a missing ownership predicate may be justified. It deserves a better explanation than “implemented best practices.”
Then follow data from entry point to side effect. For request-handling code, identify the user-controlled input, validation, authorization decision, database or network call, output, and logs. The OWASP Secure Code Review Cheat Sheet is useful here because it emphasizes business logic and trust boundaries that automated tools cannot settle alone.
Check the boundaries the prompt did not mention
AI fixes often optimize for the named path. Review the neighboring paths a real user or attacker can take.
For an authorization fix:
- Another user in the same team
- A user in a different team
- An administrator or support role
- A missing, malformed, or deleted resource
- A bulk, export, or download route using the same record
For a form or API fix:
- Empty values
- Duplicate submission
- Maximum size
- Unicode and unusual whitespace
- Slow upstream response
- Cancellation and retry
For a UI state fix:
- Navigation before the request completes
- Two quick clicks
- A stale tab
- A failed request after optimistic state changed
- Keyboard and screen-reader behavior
The goal is not to invent infinite edge cases. It is to test the trust boundary and failure mode, not only the demo.
Make the test fail on the old code
A regression test earns its name only if it catches the regression.
When practical:
- Add or inspect the test.
- Run it against the vulnerable or broken behavior and see it fail for the expected reason.
- Apply the fix.
- Run it again and see it pass.
If the editor changed both production code and an existing test, read the assertion carefully. Watch for a narrower input, removed negative case, updated snapshot that accepts the bug, or a mock that bypasses the real boundary.
Tests written by the same system are still useful. They are not independent by default.
Run the repository’s checks from a clean state
Run the narrow test first for fast feedback, then the checks the repository uses as a merge gate.
That may include:
- Type checking and linting
- Unit and integration tests
- Production build
- Migration validation
- Dependency or source audit
- Browser test for the affected flow
Do not accept “tests should pass” from the editor. Read the current output after the final code change.
For a dependency or generated-file change, reproduce the install or generation from the committed inputs. A working long-lived development folder can hide missing lockfile entries and stale artifacts.
Verify the behavior at the layer that found it
If a live-site scan found a missing header, unit tests are not the final proof. Deploy to a safe environment and inspect the response.
If the bug appeared in a browser flow, complete that flow with the production build. If Search Console reported a canonical problem, inspect the rendered public page. If a source audit found a secret-shaped value in a client bundle, search the new bundle.
The verification should meet the bug where it lived.
SiteCMD’s fixing-with-AI workflow uses the same idea: send a concrete finding to the editor, make the change, then run the scanner again to see whether the evidence cleared. The AI-generated code audit checklist covers the recurring security and failure patterns worth reviewing across a larger generated change.
Know when to stop the merge
Pause when:
- You cannot explain why the fix works.
- The patch changes a trust boundary without a negative test.
- A failing test was deleted or diluted.
- New dependencies or privileges appeared without need.
- The fix catches every error and returns a plausible empty value.
- Verification requires production access you do not have.
- The diff is too broad to review as one decision.
Ask the editor to split the patch, add the missing evidence, or revert and try a narrower approach. Speed is valuable only while the change remains understandable.
The merge checklist
- Original failure is stated independently of the patch
- Diff scope and every changed test were reviewed
- Trust boundaries and negative cases were exercised
- Regression test fails without the fix and passes with it
- Repository checks ran after the final edit
- Behavior was verified at the layer that originally failed
- A future recurrence will be observable
AI can write the patch and help write the proof. It cannot make the proof independent just by saying “done.”