Decide the agent's authority
Type the steps of a workflow. For each one, say how hard a wrong action is to undo, whether two good engineers could disagree, and what a second run does. The rubric then names who owns the step: code, an agent, or an agent that suggests while a human approves.
4 undo levels · 5 questions per step · 3 reference examples · 40 minutes
Built by Sunil Mathew, co-authored with Claude
From The Living Craft, practical learning and consulting with Sunil Mathew. Explore the programmes
- You have a workflow, and someone wants to hand part of it to an AI agent. This tool decides which steps the agent may own, which it may only suggest on, and which stay as plain code.
- It works one step at a time. For each step you answer three questions: is there one right answer, how hard is a wrong action to undo, and what happens if the step runs twice.
- The rubric turns those three answers into an owner for the step: code, an agent, or an agent that suggests while a human approves. The owner appears beside the step as you answer.
- The result is an authority map for the whole workflow: how many steps the agent acts on alone, how many it only suggests on, and which steps cannot go further yet.
- Pick one workflow you own. A runbook, a support flow, a release path. Not the most interesting one, the one you know best.
- Type each step of the workflow into the sheet, one per row. Decide nothing yet. The first guess about owners is usually wrong and it biases everything after it.
- For every row, fill Evidence and Undo cost first. Only then answer Judgment and Repeat.
- Watch the owner beside each step and the tally in the bar at the top. Both update as you answer.
- Read the result at the end. Download the review as an Excel workbook to run it on the next workflow. You can also copy the sheet, print it, or get the PDF of your assessment.
Give every action a level before you decide who owns the step.
Undo cost is the price of a wrong action, not the chance of one. Ask: if this step runs wrongly at 3 pm on a Tuesday, what does it take to put things back, and who sees it happen? The answer is a level from R0 to R3. It has nothing to do with how good the model is.
It is a separate limit from model quality, and you have to set both. How good the model is decides how much thinking you hand over. How hard an action is to undo decides how much authority you hand over. A demo that finds the cause of an outage in ninety seconds proves the first. It says nothing about the second.
The line is almost always between R1 and R2. Below the line, a wrong action is fixed inside the team and the customer never knows. Above it, somebody outside the team sees the mistake, or nobody can fix it at all. Above the line the agent only suggests. Below it the agent may act.
Score the action, not the step name. "Roll back the deploy" sounds like one action. A rollback that also reverses a database migration is a different action with the same name, and a different level. If one step hides two actions with two levels, split it into two rows.
Read a log. Add a few servers. Draft a message. Produce a list of possible causes.
Turn a config flag off. Put a job back in the queue. Restart a pod. Revert a small, clean commit.
Roll back a deploy that changed data. A support message already sent. An alert that was never raised.
A payment sent. A database migration run. A record deleted. A public API released. Something a customer has already been told.
Ask them of every step. Never of the workflow as a whole.
Each question fills one column of the sheet. Three of them decide the owner: judgment, undo cost and repeat. The other two, evidence and hard limit, are what you write down so that the owner can be held to it.
Evidence
What must be true before this step runs, and can you confirm it was checked rather than guessed?
If the input was invented by an earlier step, every check after it still passes. Those checks are then only for show.
Judgment
Is there one right answer, or can two good engineers disagree?
If they can disagree, you need an eval set (a set of real cases with known-correct answers) before a model is safe here. If they cannot, you need a rule in code, not a model.
Hard limit
What must not bend here, however good the explanation sounds?
Write it as a fixed check in code, placed where no argument can go around it. A limit that lives in a prompt is a suggestion.
Undo cost
If this step acts wrongly, is it R0, R1, R2 or R3?
This is the score. It decides how much authority the step can carry, whatever the model can do.
Repeat
If this step runs a second time because nobody learnt the result of the first run, what happens?
A tool call that times out instead of failing is the most common agent incident. The system does not know whether the action happened, retries, and does it twice. The second action usually does more damage than the first.
One row per step. Fill Evidence and Undo cost first.
The authority map, read against the rubric.
Who owns each step
Before you hand anything over
You have examined one operating decision: whether the agent holds permission at the moment it executes an action. Build the wider reasoning through practical work and review in a Living Craft programme. Explore the programmes
How three answers become one owner.
Apply it only once a row is full, never while filling it. The count beside each rule is how many of your steps it currently decides.
May: Pass or fail. A rule, with no model in the path.
A rule is cheaper, faster and easier to audit than a model. Do not spend a model on a check that a simple condition can do. Most rows land here. That is a good result, not a disappointing one.
May: Act on its own, once the eval set exists.
This is where an agent earns its place: real judgment is needed, and a wrong call is cheap to undo. Build the eval set before you build the agent, not after.
May: Prepare the action and the reasoning. A named person approves.
"Not yet" means until you have a log of real production failures for this step. It does not mean until the demo looks good.
May: Nothing, until you can say what a second run does.
Handling a repeat is not a phase-two item. It is the difference between one failover and two. Find out what a second run does before you give this row to anyone.
Three workflows with every row already decided.
Each table shows the author's own call for every step. Load one into the sheet to see the owner the rubric gives beside it, and change an answer to see the owner move.
Incident triage and fixing
An agent watches alerts, works out the likely cause of an outage, and proposes or runs a fix.
| Step | Evidence needed | Judgment | Hard limit | Undo | Repeat | Author's call | What it may do | If evidence is missing |
|---|---|---|---|---|---|---|---|---|
| Collect signals | Deploy log, alerts, health of dependencies, service owner list | One right answer | Read only. No write to any system. | R0 | Safe or detected | Code | Read only | Say which signal is missing and lower the confidence. Do not fill the gap. |
| Line up the timeline | Deploy and alert timestamps that were actually read | One right answer | Never narrow the time window on partial data. | R0 | Safe or detected | Code | Give a likely time window | Widen the window. Never narrow it on partial data. |
| Rank possible causes | The signals above, with evidence attached to each cause | Engineers can disagree | Every cause shows its evidence. "Not enough evidence" is always an allowed answer. | R0 | Safe or detected | Model | A ranked list, each item showing its evidence, plus a clear "not enough evidence" option | Return "not enough evidence" as a proper answer. |
| Decide whether to page someone | Severity rules, owner, customer impact signal | Engineers can disagree | The agent may ask for a page. It may never stop one. | R2 | Safe or detected | Human (rule in code) | The agent may ask for a page. It may not stop one. | Page by default. |
| Suggest a fix | The chosen cause, what it will affect, and the command to reverse it | Engineers can disagree | No plan without a way back. | R0 | Safe or detected | Model | Write the plan, including how to undo it | Refuse to suggest an action that has no known way back. |
| Act: add capacity or fail over inside a tier | Plan approved, run ID created, earlier run checked | Engineers can disagree | Never act when the result of the earlier run is unknown. | R1 | Safe or detected | Model / Code | Act | Do not act when the earlier state is unknown. Check first. |
| Act: roll back a deploy with a migration, change a Tier-0 service, or act during a freeze | All of the above, plus a human approval | Engineers can disagree | A human runs it. Always. | R3 | Safe or detected | Human | Agent prepares the command. Human runs it. | Escalate. |
| Confirm and close | Signals recovered and stayed recovered for a fixed period | One right answer | Never close an incident because the alerts stopped. | R1 | Safe or detected | Code + Human | Confirm | Keep the incident open. |
Where the author says no agent, not yet, and why
Stopping a page. This is the only decision here that gives you no feedback. If the agent pages too often, everyone notices. If it wrongly decides not to page, nobody ever finds out. You cannot build an eval set from outcomes you never saw. So until you can measure the misses, the agent only suggests and the rule decides.
Any action above R1. The risk is not an agent that thinks badly. It is an agent that thinks well about the cause, then runs an action whose side effects were never part of what it looked at. "Roll back the deploy" sounds like one action. A rollback that also reverses a database migration is a completely different action with the same name.
Refunds and goodwill credit
A support agent reads a complaint, checks the order, and decides whether money goes back to the customer.
| Step | Evidence needed | Judgment | Hard limit | Undo | Repeat | Author's call | What it may do | If evidence is missing |
|---|---|---|---|---|---|---|---|---|
| Check the order, the payment status and past refunds | Order system and ledger, read directly | One right answer | No refund on an unconfirmed state. | R0 | Safe or detected | Code | Read only | Stop. No refund on an unconfirmed state. |
| Understand what the customer is saying | Chat or call transcript, order history | Engineers can disagree | A claim is never treated as a confirmed fact. | R0 | Safe or detected | Model | Summarise the complaint, and clearly separate what the customer claims from what we have confirmed | Mark it as unconfirmed and carry on. A believable story is not proof. |
| Check refund policy | The policy version that applied on the purchase date | One right answer | Not eligible means no refund. No story changes that. | R0 | Safe or detected | Code | Pass or fail | Not eligible means no refund. |
| Decide the amount, within a set band | Eligibility, customer tier, earlier credits given | Engineers can disagree | Never above the band. The cap is in code, not in the prompt. | R3 | Safe or detected | Model under a hard cap | Suggest an amount inside the band | Suggest the lowest amount and flag for review. |
| Decide an amount above the band, or any exception to policy | All of the above | Engineers can disagree | The policy check never bends. | R3 | Safe or detected | Human | Agent writes the reasoning. Human decides. | Escalate. |
| Send the refund | Approved amount, plus a unique key made from order ID and reason code | One right answer | Send once. Never send when the result of the earlier attempt is unknown. | R3 | Safe or detected | Code | Send once | Check the earlier attempt first, then act. |
| Record what was decided | Decision, amount, reason code | One right answer | Every decision is logged, including the ones a human made. | R1 | Safe or detected | Code + weekly human review | Log it | Log what is known and mark the gap. |
Where the author says no agent, not yet, and why
Amounts above the band. An agent that is rewarded for closing tickets quickly will learn to give money away to make complaints go away. You will not see this in the transcript, which reads very well. You will see it in next quarter’s refund numbers. This is not a guardrail problem. It is a problem with what you asked the agent to optimise for, and better prompts will not fix it.
Exceptions to policy. The eligibility check is the one thing here that must not bend. The moment a good enough story can create an exception, the policy has stopped being a control and become a suggestion.
The record step. Most teams skip it. Every goodwill refund teaches customers what to ask for next time. This is the row that adds up over months.
Automatic pull request merge
An agent reviews a pull request, decides whether it is safe, and merges it or asks for a person.
| Step | Evidence needed | Judgment | Hard limit | Undo | Repeat | Author's call | What it may do | If evidence is missing |
|---|---|---|---|---|---|---|---|---|
| Run CI, tests and static analysis | Green build on the merge commit, not on the branch head | One right answer | Green on the merge commit. A green branch head does not count. | R0 | Safe or detected | Code | Pass or fail | Fail closed. |
| Compare what the code does with what the PR says it does | The diff, the description, the linked ticket | Engineers can disagree | A mismatch is always a flag, never a fix. | R0 | Safe or detected | Model | Flag it when the two do not match | Flag as "cannot review". |
| Check owners and approvals | CODEOWNERS file, approval records | One right answer | No approval, no merge. | R0 | Safe or detected | Code | Pass or fail | Fail closed. |
| Merge a low-risk change | No migration, no auth or payment code, no public API change, and the revert is one clean commit | Engineers can disagree | Only when the revert is one clean commit. | R1 | Safe or detected | Model / Code | Merge | Do not merge. Ask for a review. |
| Merge anything touching auth, payments, migrations or a public API | All of the above | Engineers can disagree | A human merges. Always. | R3 | Safe or detected | Human | Agent prepares and summarises. Human merges. | Escalate. |
| Trigger the deploy | Merge done, deploy window open | Engineers can disagree | The deploy is its own decision with its own owner. Never bundled with the merge. | R3 | Safe or detected | A separate decision, with its own owner | Nothing here. The row exists so that the deploy has its own owner. | Do not deploy. |
The original note scores the deploy trigger R2–R3. The sheet takes one level per step, so it loads as R3, the worse case.
Where the author says no agent, not yet, and why
Anything touching auth, payments, migrations or a public API. Once a migration has run or an API is public, undoing it is not the reverse of doing it. Also, judging whether the code matches the description is exactly the task where models sound most confident and are most often wrong. Both the description and the diff read well on their own. The gap between them is the bug.
The deploy trigger. Keep it as its own row with its own owner. Teams that merge and deploy in one step have given away deploy authority without ever discussing it. This is the most common accidental handover I see, and it never shows up in an architecture diagram, because it lives in a CI config file.
Pick the workflow you are most tired of.
One workflow or runbook that you own. Not the most interesting one.
- List the steps. Decide nothing.
Do not assign owners yet. Your first guess will be wrong and it will bias everything after it.
- Fill only two columns: Evidence and Undo cost.
Every row, before anyone says the word "agent".
- Mark the rows where two good engineers could disagree.
Judgment is the axis that decides whether a model belongs here at all.
- Read the owner the rubric gives each row.
Only now. Reading owners while you fill the table is how the table ends up agreeing with whatever you already thought.
- Circle every row where you cannot answer question 5.
Those are the rows that turn one failover into two.
What you will usually find: fewer agent rows than you expected, several rows that should have been plain code all along, and at least one action that cannot be undone and has never had a named owner.
That last one is worth the forty minutes on its own.
This tells you where an agent belongs. Not whether yours works.
For every row where two engineers could disagree, you now need an eval set. Writing evals for those cases is a different skill from the assertion tests you already know, and it is the skill that separates a demo from something you can leave running.
The Living Craft's Architecting Agentic AI Systems programme is built on exactly this: building agents for decisions where the right answer is genuinely debatable, and building the evals that show they hold up. 30 live hours with Sunil Mathew; confirm complete setup and pre-session preparation before joining. For experienced engineers, architects and engineering leaders.
Decide where the agent may act and where people retain authority. Explore Architecting Agentic AI Systems: 30 live hours with Sunil Mathew; confirm complete setup and pre-session preparation before joining. Request a fit call at https://learning.thelivingcraft.ai/programmes/agentic-systems/architecting-agentic-ai-systems/#discuss-architecting-agentic-ai-systems
Published 14 September 2026 · free to use and to pass on