How to Verify AI-Generated Code Before You Ship
A five-layer check for requirements, behavior, regressions, security boundaries, and an honest completion handoff.
Verify the requirement, capture the complete change, reproduce the critical behavior, challenge the author’s assumptions, run repository-wide deterministic checks, inspect security-sensitive boundaries, and record what remains unknown. A passing test is evidence for one assertion, not proof that the task was understood.
1. Restate the behavioral contract
Before reading the patch, write down what must change, what must stay unchanged, and the failure case that would be expensive. This prevents an elegant implementation of the wrong task from defining its own acceptance criteria.
2. Capture the complete candidate change
Inventory staged, unstaged, deleted, and untracked files. Check generated artifacts and migrations deliberately. If a file is excluded because it is secret, too large, or outside the workspace, record that limit.
3. Reproduce one consequential claim
Run the smallest test that can disprove the implementation: retry a payment callback, exercise an authorization boundary, mutate state after a cached result, or build the route that previously failed only in production. Prefer observable behavior over a prose explanation.
For example, if an agent changes invoice retries, do not stop at a unit test that calls the helper once. Send the same idempotency key twice, inspect the stored state, and confirm the second response is stable. The example is deliberately small: a narrow reproduction often exposes a wrong assumption faster than a broad test suite.
4. Separate author checks from independent checks
The author should lint, typecheck, test, and review while working. Then a fresh reviewer should challenge task interpretation and test adequacy without treating the author transcript as evidence. Independence is a workflow property; it is still not a guarantee of correctness.
5. Keep deterministic and human layers
- 1Author loopDid the implementation reach a coherent candidate finish?
- 2Independent passDoes the completion claim survive a fresh reconstruction and targeted challenge?
- 3CI and scannersDo encoded builds, tests, types, dependencies, and security rules pass?
- 4Human or policy ownerAre the product, architecture, and risk tradeoffs acceptable?
Keep an evidence ledger
For each important claim, record the command or fixture used, the observed result, and the environment that produced it. Link findings to files and lines where possible. This makes a later repair pass faster and prevents a stale green result from surviving after the workspace or dependency graph changes.
The ledger does not need to be elaborate. A short table in the change record—claim, reproduction, result, limitation, owner—is enough to show which parts were verified and which still rely on judgment.