The Run-Cost Model Tool
Put a number on an agent proposal before anyone commits to building it. Four ways of doing the same job, costed side by side in seven short steps, with the operating lines most business cases leave out. The number it lands on is cost per acceptable outcome.
4 options · 7 sections · 83 lines · reference example loaded
Built by Sunil Mathew, co-authored with Claude
From The Living Craft, practical learning and consulting with Sunil Mathew. Explore the programmes
The reference example is loaded. An ordering agent across 40 sites in India: 2,000 purchase orders a month, every figure in rupees at Indian rates. Every one of the 83 lines is filled, so you can see what a finished model looks like before you type anything. Change any figure and the model becomes yours.
- You are asked to put a number on an AI agent proposal. This tool gives you that number, and it includes the costs most business cases leave out.
- Four options are compared side by side over one period: a rules workflow as first written, the same workflow rebuilt on the decision rules the agent build forced the team to write down, a model that drafts while a person approves, and a full agent with tools.
- Build cost is kept apart from run cost. Quality sits beside cost, so the number you argue about is cost per acceptable outcome, not cost per case.
- The result names the option with the lowest cost per acceptable outcome, says whether each option ever pays back its build, and lists the assumptions to check before you trust it.
- The reference example is already loaded: an ordering agent across 40 sites, every line filled. Change any figure to make the model yours, or start from a blank model.
- Work through the seven sections with Next and Back. Sections 1 and 2 take one value each. Sections 3 to 7 take one value per option.
- Leave a line blank if you do not know it. A blank keeps the result off the screen; a 0 is a claim. After you fill a blank line, the page moves to the next blank line for you.
- Watch the bar and the progress bar at the top. The bar shows cost per acceptable outcome for each option as soon as enough lines are filled.
- Read the result at the end. It names the leading option, the break-even month for each, and the assumptions that decide the answer. Get the PDF of your model there.
- Prefer a spreadsheet? Download Excel version, on the Start step, has the same rows and the same formulas, with an instruction beside every line.
Plain code that follows fixed rules, written from what the team already knew. No model anywhere in the path. It handles the common cases and declines or escalates the rest, because the rules for the hard cases were never written down.
The same kind of plain code, rebuilt after the agent was built. Building the agent forced the team to write every decision rule down. Code rebuilt on that written spec covers more cases and declines fewer, so it costs more to build and less to run. Still no model in the path.
A model drafts a recommendation. A person approves every one before it takes effect.
A model plans, calls tools and acts. People review a sample of cases and the exceptions.
Rules and Rules v2 are both plain code with no model in the path. The difference is where the rules came from. Rules runs on the rules the team already knew. Rules v2 runs on every rule that got written down while the agent was being built, which is why it handles more cases and costs more to build. If you have not done that spec work, set the Rules v2 column equal to Rules, or leave it blank.
Five rules that decide whether the number is evidence or a guess.
- Fill it from a trial, not from a guess. Escalation share, review minutes and the retry multiplier come from logs of real cases. A demo produces none of them. Nothing here is a benchmark: there is no industry number for your escalation rate.
- Argue about cost per acceptable outcome, not cost per case. Cost per case rewards a system that quietly declines the hard cases. Cost per acceptable outcome charges it for them.
- One definition of an acceptable outcome, one reviewer, all four options. The acceptable-outcome rate must come from one written definition, applied to every option by the same person. Otherwise the comparison is theatre.
- Blank means unknown. Zero means measured. A blank keeps the result off the screen until you know the number. Type a 0 only when you have measured a 0. "This system never retries" is a claim.
- Change the rows, keep the shape. Add a cost line your workload has and this model does not: data labelling, a vendor licence, a second reviewer for high-value decisions. The shape is the point: build kept apart from run, quality beside cost, and the operating lines visible.
Workload and manual baseline
What does this work cost today, done by hand?
Every option is measured against this. If the manual baseline is wrong, every break-even month below it is wrong by the same amount.
Such as INR or USD. Shown in front of every money figure. Not counted as a line.
One case is one unit of work: a purchase order, a ticket, a refund request. Take the count from last quarter, not from the plan.
The window the build has to pay for itself. Twelve is the usual budget horizon. Make it longer only if you can defend the longer horizon in the room.
How long one person takes to finish one case today. Time it on ten real cases.
Fully loaded: salary, benefits, seat. Use the rate of the people who actually do the work today.
Cases per month multiplied by months.
Total cases, times minutes per case, times the hourly rate. This is the number every option has to beat.
Team rates
What does an hour of each person cost?
Three rates, shared by all four options. Engineers build and handle escalations; reviewers approve output and finish declined cases.
What one engineer-day costs you. Used for the build and for upkeep.
Used for escalations: the cases an engineer has to look at.
Used for reviewing output and for finishing declined cases by hand. Often the same rate as the manual baseline.
Build cost, paid once
What does each option cost before it handles its first case?
Engineering days, converted at the day rate. Evaluation work is a separate line because it is the one most estimates skip, and it is where a model-based option earns the right to be trusted.
From the first line of code to a system that handles real cases. Include integration with the tools it calls.
An eval is a set of real cases with known-correct answers that the system is scored against. Building one is engineering time.
Build days plus eval days, times the engineering day rate.
Inference and tool calls, per month
What does the model bill actually include?
Inference is the cost of calling the model. Most business cases price the successful calls and stop there. The retry multiplier is where the rest goes.
How many times a case calls the model when everything works. An agent that plans, then acts, then checks its work makes several.
Total model calls divided by successful ones. Count timeouts, tool errors and loops that re-enter, not only refusals. 1.0 means the system never retries, which no real system does.
Average cost of one call, including the input tokens. Take it from a month of real invoices, not from the price list.
Lookups, searches, writes to other systems. Anything with a price per call or a rate limit.
Average across the tools it calls. Internal calls are not free: price the compute.
Cases per month, times calls per case, times the retry multiplier, times the cost per call.
Cases per month, times tool calls per case, times the cost per tool call.
Human in the loop, per month
How much human time does each option still need?
Three kinds of human time. Review is routine checking of output. Escalation is a case that needs an engineer. Declined is a case the system would not finish, which a person then does by hand. This section decides most comparisons.
100 for a model-assisted draft, because a person approves every one. A sample, such as 20, for a full agent.
Time it on real cases. Approving a good draft is quick; checking an agent that acted on its own is not.
Cases that stop and need somebody who can read the trace. Measure it in a trial. A demo will show you 0.
Reading the trace, working out what happened, fixing the case. Longer for an agent, because there is more trace to read.
Cases the system refuses or gives up on. They do not disappear: a person finishes them, at the manual rate.
Usually close to the manual baseline, because the person starts from nothing.
Cases per month, times the share reviewed, times minutes per review, at the reviewer rate.
Cases per month, times the share escalated, times minutes per escalation, at the engineer rate.
Cases per month, times the share declined, times manual minutes, at the reviewer rate.
Upkeep and operations, per month
What does it cost to keep the system trustworthy after launch?
A rules workflow needs almost none of this. A model-based option needs all of it, every month, and none of it appears in a demo.
Tracing, logging, the eval runner, the dashboards. Licence fees or the compute to run your own.
Engineering days a month spent adding cases to the eval set and fixing the ones that go stale.
Engineering days a YEAR spent re-running the eval set after a model or SDK update. Spread across the months by the model. Count every update the provider ships.
Engineering days a month spent checking that a prompt change did not break something that used to work.
Engineering days a month on incidents caused by the system. Take it from the on-call log.
Tooling, plus every engineering day above at the day rate, with the yearly line divided by twelve.
Quality
How often does each option produce an outcome you would accept?
This is what turns cost per case into cost per acceptable outcome. One written definition of "acceptable", applied to every option by the same reviewer.
The share of cases that meet the written definition with no rework. Measure it on the same set of cases for all four options.
Total cases, times the acceptable-outcome rate.
The outcome, read against the rubric.
The reference example is loaded. These are the example's numbers, not yours.
The agent build paid for itself by forcing the team to write the decision rules down. Ship the rebuilt rules workflow, and keep the specification under version control like any other code. Revisit the model when volume or the price of a call changes.
- Cost per case picks Rules workflow, as first written. Cost per acceptable outcome picks Rules v2, rebuilt on the written spec. Argue about the second one: cost per case rewards a system that quietly declines the hard cases.
- Full agent with tools breaks even in month 17.7, after the 12-month period you modelled. It pays back, but only if you can defend the longer horizon.
| Line | RulesPlain code, from the rules the team already knew. | Rules v2Plain code again, rebuilt on every rule the agent build forced you to write down. | AssistedModel drafts. A person approves each one. | AgentModel plans, calls tools, acts. |
|---|---|---|---|---|
| Run cost per monthModel, tools, review, escalation, declined cases and upkeep, added up. | INR 50,467 | INR 37,733 | INR 1,42,418 | INR 3,67,967 |
| Working for Run cost per month. Rules model INR 0 + tools INR 200 + review INR 14,400 + escalation INR 16,667 + declined INR 19,200 + upkeep INR 0 = INR 50,467 Rules v2 model INR 0 + tools INR 200 + review INR 9,600 + escalation INR 13,333 + declined INR 9,600 + upkeep INR 5,000 = INR 37,733 Assisted model INR 4,968 + tools INR 1,000 + review INR 40,000 + escalation INR 31,250 + declined INR 7,200 + upkeep INR 58,000 = INR 1,42,418 Agent model INR 40,500 + tools INR 5,000 + review INR 16,000 + escalation INR 1,16,667 + declined INR 4,800 + upkeep INR 1,85,000 = INR 3,67,967 | ||||
| Run cost for the periodRun cost per month, times months. | INR 6,05,600 | INR 4,52,800 | INR 17,09,016 | INR 44,15,600 |
| Working for Run cost for the period. Rules INR 50,467 × 12 = INR 6,05,600 Rules v2 INR 37,733 × 12 = INR 4,52,800 Assisted INR 1,42,418 × 12 = INR 17,09,016 Agent INR 3,67,967 × 12 = INR 44,15,600 | ||||
| Total cost for the period, build plus runThe build, paid once, plus the run cost for the period. | INR 11,05,600 | INR 11,52,800 | INR 27,29,016 | INR 67,55,600 |
| Working for Total cost for the period, build plus run. Rules build INR 5,00,000 + run INR 6,05,600 = INR 11,05,600 Rules v2 build INR 7,00,000 + run INR 4,52,800 = INR 11,52,800 Assisted build INR 10,20,000 + run INR 17,09,016 = INR 27,29,016 Agent build INR 23,40,000 + run INR 44,15,600 = INR 67,55,600 | ||||
| Cost per caseTotal cost divided by total cases. The wrong number to argue about on its own. | INR 46.07 | INR 48.03 | INR 113.71 | INR 281.48 |
| Working for Cost per case. Rules INR 11,05,600 ÷ 24,000 = INR 46.07 Rules v2 INR 11,52,800 ÷ 24,000 = INR 48.03 Assisted INR 27,29,016 ÷ 24,000 = INR 113.71 Agent INR 67,55,600 ÷ 24,000 = INR 281.48 | ||||
| Cost per acceptable outcomeTotal cost divided by acceptable outcomes. The number to argue about. | INR 59.06 | INR 52.78 | INR 127.76 | INR 299.45 |
| Working for Cost per acceptable outcome. Rules INR 11,05,600 ÷ 18,720 = INR 59.06 Rules v2 INR 11,52,800 ÷ 21,840 = INR 52.78 Assisted INR 27,29,016 ÷ 21,360 = INR 127.76 Agent INR 67,55,600 ÷ 22,560 = INR 299.45 | ||||
| Net position against the manual baselineManual baseline minus total cost. Positive means the option saves money over the period. | INR 48,94,400 | INR 48,47,200 | INR 32,70,984 | INR -7,55,600 |
| Working for Net position against the manual baseline. Rules manual INR 60,00,000 − total INR 11,05,600 = INR 48,94,400 Rules v2 manual INR 60,00,000 − total INR 11,52,800 = INR 48,47,200 Assisted manual INR 60,00,000 − total INR 27,29,016 = INR 32,70,984 Agent manual INR 60,00,000 − total INR 67,55,600 = INR -7,55,600 | ||||
| Months to break even on the buildBuild cost divided by the monthly saving against doing it by hand. "Never" means the option costs more to run each month than the manual process. | 1.1 months | 1.5 months | 2.9 months | 17.7 months |
| Working for Months to break even on the build. Rules build INR 5,00,000 ÷ (manual INR 5,00,000 − run INR 50,467 per month) = 1.1 months Rules v2 build INR 7,00,000 ÷ (manual INR 5,00,000 − run INR 37,733 per month) = 1.5 months Assisted build INR 10,20,000 ÷ (manual INR 5,00,000 − run INR 1,42,418 per month) = 2.9 months Agent build INR 23,40,000 ÷ (manual INR 5,00,000 − run INR 3,67,967 per month) = 17.7 months | ||||
Check these assumptions first
The leading option first, then the full agent, then the rest. Each one is a line that decides the answer and is usually guessed.
- 01 Rules v2, rebuilt on the written spec: escalations are 35% of run cost. That line decides the comparison. Measure it in a trial before you trust it.
- 02 Full agent with tools: upkeep is 50% of run cost. That line decides the comparison. Measure it in a trial before you trust it.
- 03 Rules workflow, as first written: declined cases are 38% of run cost. That line decides the comparison. Measure it in a trial before you trust it.
- 04 Model-assisted draft: upkeep is 41% of run cost. That line decides the comparison. Measure it in a trial before you trust it.
You have examined one operating decision: what the system may spend, and how it recovers when it reaches that budget. Build the wider reasoning through practical work and review in a Living Craft programme. Explore the programmes
Agree this before the numbers go in, not after.
The leading option is the one with the lowest cost per acceptable outcome among the options that save money against the manual baseline over the period. The row that matches your result is highlighted.
Three flags sit beside the outcome and do not change it. Disagreement: cost per case and cost per acceptable outcome pick different options. Never breaks even: an option costs more to run each month than doing it by hand. Beyond the period: an option pays back, but after the months you modelled.
The model bill is usually the smallest line.
Each of these is marked often left out in the model above. Together they are most of what it costs to run an agent, and none of them appears in a demo.
Retry and failure multiplier
Total model calls divided by successful ones. Count timeouts, tool errors and loops that re-enter, not only refusals. 1.0 means the system never retries, which no real system does.
Review minutes per reviewed case
Time it on real cases. Approving a good draft is quick; checking an agent that acted on its own is not.
Engineer minutes per escalation
Reading the trace, working out what happened, fixing the case. Longer for an agent, because there is more trace to read.
Share of cases declined or abandoned
Cases the system refuses or gives up on. They do not disappear: a person finishes them, at the manual rate.
Observability and eval tooling
Tracing, logging, the eval runner, the dashboards. Licence fees or the compute to run your own.
Eval maintenance
Engineering days a month spent adding cases to the eval set and fixing the ones that go stale.
Model-version re-qualification
Engineering days a YEAR spent re-running the eval set after a model or SDK update. Spread across the months by the model. Count every update the provider ships.
Prompt and regression testing
Engineering days a month spent checking that a prompt change did not break something that used to work.
Incident and on-call load
Engineering days a month on incidents caused by the system. Take it from the on-call log.
A number you can take to a budget conversation. Not the whole argument.
- It assumes steady volume, one workload and stable rates for the whole period.
- It does not price revenue upside. A system that wins new business is not captured by a cost line.
- It does not price the risk of a wrong action that cannot be undone. That belongs in the Agent Authority Review, not in a cell here.
- It does not price the option value of the capability you build. If that is the argument, write it in the decision record in words, not as a number in this sheet.
An agent proposal is also a commitment to operate an agent. This tool turns that sentence into a number. Sometimes the number says build it. The reference example is a case where it did not, and where the build was still worth doing, because it produced the written specification that made the cheap option good.
Connect the cost of accepted work to the architecture you choose. Explore Architecting Agentic AI Systems: 30 live hours with Sunil Mathew; confirm complete setup and pre-session preparation before joining.
Connect the cost of accepted work to the architecture you choose. Explore Architecting Agentic AI Systems: 30 live hours with Sunil Mathew; confirm complete setup and pre-session preparation before joining. Request a fit call at https://learning.thelivingcraft.ai/programmes/agentic-systems/architecting-agentic-ai-systems/#discuss-architecting-agentic-ai-systems
Published 14 September 2026 · free to use and to pass on