AI-generated code is ready to ship only when evidence proves the intended user outcome in the final environment. An agent's summary, a clean diff, and a passing unit test are useful signals, but none is a universal definition of done.
The practical answer is an evidence ladder: verify the source, the generated artifact, the integrated runtime, the user action, and the delivered result.
Start with the outcome
Write one observable sentence:
After this change, the user can complete X and receives Y.
Then list the invariants: what must remain unchanged in URLs, permissions, data ownership, pricing, state, or existing behavior.
Without this step, verification drifts toward what is easy to test instead of what matters.
Use the evidence ladder
- Static validity: types, syntax, lint, schema, and configuration parse correctly.
- Focused behavior: the changed function or component passes its narrow tests.
- Integration: registration, dependency injection, routing, persistence, and error paths work together.
- Generated output: templates, scripts, metadata, and configuration are checked after generation.
- User journey: the real action succeeds in a representative rendered environment.
- Delivery: the expected release, URL, event, or artifact is available from its final consumer.
Climb only as high as the risk and impact require, but never stop below the layer the user actually consumes.
Test the final generated artifact
Many agent errors hide between source and output:
- a template emits invalid JavaScript;
- Markdown syntax appears literally on a page;
- a route exists in a module but is not registered;
- a valid config references a missing script;
- metadata updates while the page body remains stale.
The verification target is whatever the browser, runtime, store, or customer receives—not only the source template.
Add negative checks
A happy path does not prove boundaries. Test the failures a real user can encounter:
- invalid input;
- no permission;
- stale state;
- duplicate retry;
- upstream timeout;
- empty data;
- mobile layout;
- old cache.
For an idempotent action, retry the same identity and prove it lands once. For a last-known-good page, simulate upstream failure and prove the previous version remains available.
Separate builder and reviewer evidence
The builder should report commands and known gaps. A reviewer should independently compare the original goal with the actual diff and outputs.
A useful review has three questions:
- Hard evidence: did the change and checks actually occur?
- Intent: does the result still match the user outcome and invariants?
- Judgment: was the chosen path proportionate and maintainable?
A Reddit build log captures the central problem plainly: treating the agent's final message as the finished task. The proposed correction was to require proof before accepting “done”: discussion.
A shipping checklist
Product result
- The target user action succeeds.
- Existing user actions still behave as intended.
- Copy and UI contain no implementation language or template residue.
Code and artifact
- Types, tests, and build pass.
- Generated output parses.
- Final route or bundle contains the change.
- Secrets are absent from browser output and logs.
Runtime and delivery
- The deployed identity matches the intended version.
- Real HTTP status, page metadata, and body agree.
- Retry and failure behavior preserve state.
- Temporary test objects are retained intentionally or cleaned.
Handoff
- Evidence is linked to exact versions or commands.
- Known gaps are explicit.
- The next session does not need to guess what shipped.
When manual verification is necessary
Use a real browser, device, or account when the feature depends on layout, focus, animation, authentication, provider state, or user perception. Automation should reduce repetition, not erase the final user's perspective.
The core principle
Do not ask whether the agent finished. Ask whether the final consumer proves the promised result—and whether the failure cases preserve it.