AI Agent Definition of Done: How to Define Completion
Reliable agents need observable completion criteria. Define tasks, outcomes, failure cases, and human calibration before treating an AI workflow as dependable.
This article synthesizes public Anthropic and Claude engineering material with Gwarket's analysis of verifiable AI workflows. It is not Anthropic documentation or a product commitment.
An agent does not become reliable because it speaks more fluently. It becomes reliable when the people using it can say what a correct result is, how to inspect it, and what must happen when it cannot reach that result.
Without that definition, every change becomes guesswork. A new prompt, model, or tool may make the output feel better one day and worse the next, with no way to tell whether the workflow actually regressed.
Define the task before evaluating the agent
Evaluation is not a final QA layer. It forces an earlier question: what are the inputs, what final state counts as correct, and which errors are unacceptable?
For a research task, “produce a high-quality summary” is too vague. A better definition might require each important conclusion to trace to a source, distinguish known facts from inference, preserve missing evidence as unknown, and leave the reader with a usable next decision.
Those criteria do not turn writing into an exam. They give both the human and the system a shared description of what the work is for.
Separate generation from verification
The simplest pattern is generator-verifier. One agent produces a result. Another checks it against stated conditions and returns concrete reasons when it fails. The pattern fits work with explicit quality criteria; it does not solve a question that even people cannot judge consistently.
The verifier does not have to be another model. Code can be tested. Data can be compared against rules. Content can be checked for sources and calibrated by an editor. The key is that the generator cannot declare success solely through its own narrative.
Three habits that make evaluation useful
- Test the final state, not only the response. Check whether the file, record, calculation, or behavior exists and is correct.
- Keep failure cases. A useful test set includes not only normal work, but also tasks where the system once overclaimed, missed a step, or should have stopped for confirmation.
- Calibrate with people. If the output serves people, the standard cannot be defined indefinitely by a model alone.
Anthropic's evaluation framework separates task, trial, trace, grader, and outcome. That separation is useful because different failures require different repairs. The agent may misunderstand the task, use the wrong source, exceed a tool boundary, or complete a correct action that the workflow failed to verify.
Avoid two kinds of false improvement
The first is a false pass: the agent learns to imitate the format while missing the point. It may add citations that do not actually support the claim.
The second is a false regression: a more cautious system appears less impressive because it now preserves uncertainty instead of hiding it. If the only metric is speed or apparent completeness, a real improvement in trust can be mistaken for decline.
You do not need a massive benchmark to begin. Start with ten real cases: five normal tasks, three common failure cases, and two situations where the correct behavior is to stop or ask. Run them when the workflow changes, then add new failures as they appear.
The point of evaluation is not a beautiful score. It is a way to improve without guessing, and to choose tools based on whether they repeatedly produce acceptable outcomes under your real constraints.