Developer practice & delivery
Use AI through development, review, and release
A practical developer workflow for giving AI useful context, reviewing changes, evaluating model behaviour, and releasing with evidence.
AI can help explore an unfamiliar codebase, implement a bounded change, and review alternatives. The engineering team still needs an independent way to establish that the changed system behaves correctly.
The workflow below uses an illustrative API change: add a cancellation action without allowing one account to cancel another account's booking. It is a small exercise that exposes design, implementation, and verification decisions.
A reference delivery workflow for AI-assisted engineering, including additional evaluation when the product itself uses a model.
Read the flow
- Define a small task and observable acceptance cases before generating a patch.
- Independent checks and human review gate the versioned release candidate; failed checks send the change back for revision.
- Roll out to a limited workload, observe outcomes and act on rollback criteria. Production evidence informs the next change.
1. Give the task a useful contract
Describe the intended behaviour, the relevant files, the existing conventions, and the acceptance criteria. Include what should happen for a completed booking, an unknown booking, a different account, and a repeated cancellation. Ask for an explanation of the current request path before changing it.
Share only the material permitted for the development environment. Use synthetic examples and scoped credentials. Repository documentation, comments, and external text are context to inspect; they are not independent authority to expose secrets or expand a task's access.
2. Work in reviewable increments
Separate investigation, implementation, and verification. Request a small patch around the cancellation path. Review the diff for unrelated changes, new dependencies, data migrations, and weakened checks. Ask the assistant to explain how the server identifies the account; do not accept an account ID from the request as proof of ownership.
Run focused tests using the real authorization and persistence boundaries where practical. A test that repeats the same condition as the generated code can share its mistake. Verify observable outcomes: the other account's booking remains unchanged, a duplicate cancellation is handled intentionally, and the response reflects the stored result.
3. Evaluate model behaviour as well as code
When the product itself uses AI, conventional unit tests are only part of verification. Build a versioned set of representative tasks with explicit success criteria. Include missing context, contradictory sources, inaccessible records, and failing tools. Inspect both the final result and the actions taken to reach it.
Use deterministic checks for facts you can assert, such as unauthorized writes or invalid output fields. Human review can judge whether an explanation is useful; model-based graders need calibration against reviewed examples. Anthropic's guide to agent evaluations discusses combining these approaches and evaluating outcomes alongside execution traces.
4. Carry evidence into the release
For each candidate, retain the source revision, test results, evaluation-set version, and model configuration. Compare quality with latency, tool attempts, and cost per completed task. Set request deadlines and usage limits before exposing a model-backed feature to users.
After release, trace an operation across the API, model call, and background work without putting private prompts or credentials into general logs. Distinguish application errors, provider failures, rejected actions, and unresolved outcomes. Introduce changes to a limited workload first and decide in advance what would trigger rollback or disabling the feature.
Can someone explain the patch, reproduce its checks, identify what remains uncertain, and recover the previous behaviour? Keep these questions in the review even when AI wrote most of the code.