Design review agenda

# Design review agenda — worked example

Refund authority for the returns assistant

Illustrative scenario, not a claimed customer incident.

The same returns assistant, reviewed at the moment it is about to be given permission to issue a refund rather than recommend one. The notes below are what the agenda looks like once the hour has been used — an agenda and its minutes are the same document, which is most of why it is worth having one.

Review: Refund authority for the returns assistant
Date: ⟨date⟩
Chair: ⟨the chair — not the author of the design⟩
In the room: ⟨design owner, payments on-call, customer operations, one reviewer from outside the team⟩
Circulated beforehand: Agent design canvas, plus the draft decision record

---

## The decision in front of us

Whether to connect the refund tool, so the assistant issues a refund rather than
recommending one to an agent who issues it.

Everything else about the system stays as it is.

## What we are not reviewing today

Not the model choice. Not the retrieval quality of the policy lookup. Not the cost of
running it. All three are real and none of them changes whether this system should be
able to move money.

## Walk one action, end to end

Walked: a customer who has already been refunded once, writing in again about the same
order with a slightly different description of it.

Noted on the way past — the second request is a different conversation, so nothing in
the model's context knows about the first refund. The only thing standing between that
and a second payment is the check inside `issue_refund`.

## Would this design have stopped the case we are worried about?

Yes, and only because the duplicate check lives inside the tool. If it had been a line
in the prompt, the second request would have paid.

**Finding.** The check reads the refund ledger. Nobody in the room could say what it
does when that read fails. ⟨owner and date⟩

## Where is each control actually enforced?

- Amount limit — enforced at the tool. It re-reads the cited clause.
- One refund per order — enforced at the tool, against the ledger.
- Only refund within the policy window — **instruction only**. Finding: this is the one
  a persuasive message gets past, and the room agreed it moves to the tool.
- Never refund an unidentified order — enforced, because the tool takes an order id and
  there is no way to call it without one.

## What happens when the evidence is not there?

Policy silent on the case → hand over. Order state ambiguous → hand over. Two policy
versions disagree → refuse and raise.

**Finding.** A wrongly refused customer currently sees the same message as an
out-of-policy one. Customer operations cannot tell the two apart, so nobody finds out
when the guard is wrong. ⟨owner and date⟩

## What could we see afterwards?

The run record holds the order, the clause and version, every tool call and its result.
It does not currently hold the refusals — only the runs that acted.

**Finding.** The refusals are the population you need to know whether the guard is too
tight. ⟨owner and date⟩

## Decisions taken, and who owns each

1. The policy-window check moves from the prompt to the tool. ⟨owner⟩
2. Refusals and hand-overs are recorded, with their reason. ⟨owner⟩
3. Wrongly refused and out-of-policy get different customer messages and different
   routes. ⟨owner, with customer operations⟩
4. Refund authority is **not** connected until 1 and 2 are done. Chair holds this one.

## What we did not settle

What the duplicate check should do when the ledger read fails — refuse, or hand over?
The room split. It needs the person who owns the ledger, who was not in the room.

## After the meeting

Design owner writes the record against the seven headings within ⟨n⟩ working days;
it goes in the repository beside the service, not in a document nobody will find.
Chair re-reads the two blocking items before authority is connected.

---

Blank version of this template · https://learning.thelivingcraft.ai/resources/templates/design-review-agenda