Template
Design review agenda
Run a review that traces one action, identifies weaknesses and leaves each decision with an owner.
The ten agenda items total sixty minutes. Adjust the time when the decision needs a longer review.
Use it when you have been asked to run a review of an agentic design, and you would rather spend the hour on the trade-offs than on excavating what was built.
By Sunil Mathew.
The blank
Copy the headings into your working document and answer each prompt. Keep unanswered decisions visible for the review.
Review: ⟨decision⟩
Date: ⟨date⟩
Chair: ⟨name⟩
In the room: ⟨participants⟩
Circulated beforehand: ⟨design and supporting records⟩
The decision in front of us
5 min
One sentence from whoever called the review, naming the change and what it would let the system do that it cannot do today.
If this cannot be said in a sentence, the review is not ready. Two decisions in one hour become one decision badly.
What we are not reviewing today
2 min
The neighbouring questions people will reach for, listed and set aside out loud so they stop arriving one at a time.
Walk one action, end to end
15 min
The design owner walks a single request from arrival to effect, out loud, against the design as it is written. Pick the case you are most worried about, not the clean one.
The room's job here is to listen and note, not to solve. Questions that start "why don't you just" are held until the four below.
Would this design have stopped the case we are worried about?
5 min
Take the case just walked and trace it through the checks as written, one at a time. Any path that still reaches the effect is a finding, and it is the most valuable output of the hour.
Where is each control actually enforced?
5 min
Go through the controls named in the design and mark each one as enforced at the tool, enforced elsewhere in the system, or expressed as an instruction to the model.
An instruction is guidance and competes with everything else in the context, including text that arrived from outside. Only what is enforced where the effect happens holds.
What happens when the evidence is not there?
5 min
Name the situations where the system cannot establish what it needs, and what each one routes to. Three outcomes are available: act, refuse, hand over with the gap named.
Include the case where your own guard is wrong. A control that silently refuses a legitimate request has swapped one failure for another.
What could we see afterwards?
5 min
Ask what the record of one run would contain, then ask the harder version: could you answer "why did this refund happen" three months later, from what is kept?
Decisions taken, and who owns each
10 min
One line per decision, each with a named owner. A decision with no owner is a discussion, and it will be renegotiated by whoever is under the most pressure.
What we did not settle
5 min
The open questions, written down as questions rather than as actions. This is the section that stops a review quietly promoting an unresolved thing into a decision.
After the meeting
3 min
Who writes the decision record, by when, and where it will live. Agree it here, in the room, while everybody can hear the answer.
The Markdown file is the blank with every prompt as a quoted block you delete as you answer it.
A completed example
Illustrative scenario, not a claimed customer incident. This example follows the returns assistant from recommendation to refund authority. Fields in ⟨angle brackets⟩ are for your own design and measurements.
Refund authority for the returns assistant
The same returns assistant, reviewed at the moment it is about to be given permission to issue a refund rather than recommend one. The example shows the agenda after the discussion, with findings and decisions recorded under the original questions.
Review: Refund authority for the returns assistant
Date: ⟨date⟩
Chair: ⟨a reviewer who is not the design author⟩
In the room: ⟨design owner, payments on-call, customer operations, reviewer from outside the team⟩
Circulated beforehand: Agent design canvas and draft decision record
The decision in front of us
Whether to connect the refund tool, so the assistant issues a refund rather than recommending one to an agent who issues it.
Everything else about the system stays as it is.
What we are not reviewing today
Not the model choice. Not the retrieval quality of the policy lookup. Not the cost of running it. All three are real and none of them changes whether this system should be able to move money.
Walk one action, end to end
Walked: a customer who has already been refunded once, writing in again about the same order with a slightly different description of it.
In this example, the second request is a different conversation with no record of the first refund in the model's context. The only thing standing between that and a second payment is the check inside
issue_refund.Would this design have stopped the case we are worried about?
In this example, the duplicate check at the tool refuses the second payment. A prompt instruction alone would not independently enforce that refusal.
Finding. The check reads the refund ledger. Nobody in the room could say what it does when that read fails. ⟨owner and date⟩
Where is each control actually enforced?
- Amount limit, enforced at the tool. It re-reads the cited clause.
- One refund per order, enforced at the tool, against the ledger.
- Only refund within the policy window, instruction only. Finding: this is the one a persuasive message gets past, and the room agreed it moves to the tool.
- Never refund an unidentified order, enforced, because the tool takes an order id and there is no way to call it without one.
What happens when the evidence is not there?
Policy silent on the case → hand over. Order state ambiguous → hand over. Two policy versions disagree → refuse and raise.
Finding. A wrongly refused customer currently sees the same message as an out-of-policy one. Customer operations cannot tell the two apart, so nobody finds out when the guard is wrong. ⟨owner and date⟩
What could we see afterwards?
The run record holds the order, the clause and version, every tool call and its result. It does not currently hold the refusals, only the runs that acted.
Finding. The refusals are the population you need to know whether the guard is too tight. ⟨owner and date⟩
Decisions taken, and who owns each
- The policy-window check moves from the prompt to the tool. ⟨owner⟩
- Refusals and hand-overs are recorded, with their reason. ⟨owner⟩
- Wrongly refused and out-of-policy get different customer messages and different routes. ⟨owner, with customer operations⟩
- Refund authority is not connected until 1 and 2 are done. Chair holds this one.
What we did not settle
If the duplicate-check ledger cannot be read, should the tool refuse or hand over? The room split. It needs the person who owns the ledger, who was not in the room.
After the meeting
Design owner writes the record against the seven headings within ⟨n⟩ working days; it goes in the repository beside the service, not in a document nobody will find. Chair re-reads the two blocking items before authority is connected.
The worked example as a Markdown file, with its illustrative label.
How to use it
Circulate the design before the meeting. Start with the Agent design canvas and add a draft decision record when available. An hour spent excavating the design is an hour not spent on the trade-offs, and everybody leaves having discussed less than they wanted to.
The chair should not be the author of the design. It is not a seniority question; it is that the author cannot both explain the thing and notice what the explanation skipped.
Two habits make the difference between this agenda and a meeting that had one. The first is that items three to seven produce findings, written down as they arrive, rather than solutions argued on the spot. The second is that every decision in item eight gets a name beside it before anybody leaves.
What this is not
It is not a sign-off, and there is no score at the end of it. The hour should expose failure paths, control locations and unresolved decisions. Tests and operational evidence still need to establish whether those controls work.
It is also not an audit of the team. The questions are about the system, and a review that drifts into whether the right person built it stops producing findings immediately.
Keep the findings after the hour
Use the Architecture decision record to preserve what the room chose, what it rejected and what remains open.
For learning designed around the decisions your team shares, discuss enterprise learning. For an outside view on one decision, see bounded advisory.