The Run-Cost Model Tool
Put a number on an agent proposal before anyone commits to building it. Four ways of doing the same job, costed side by side, with the operating lines most business cases leave out. The number it lands on is cost per acceptable outcome.
4 options · 7 sections · 83 lines · 1 reference example
Built by Sunil Mathew, co-authored with Claude
- You are asked to put a number on an AI agent proposal. This tool gives you that number, and it includes the costs most business cases leave out.
- Four options are compared side by side over one period: a rules workflow, the same rules rebuilt on a written spec, a model that drafts while a person approves, and a full agent with tools.
- Build cost is kept apart from run cost. Quality sits beside cost, so the number you argue about is cost per acceptable outcome, not cost per case.
- The result names the option with the lowest cost per acceptable outcome, says whether each option ever pays back its build, and lists the assumptions to check before you trust it.
- Set the workload and the rates first. Every other line is multiplied by them.
- Fill in each option column by column. Leave a line blank if you do not know it. A blank keeps the result off the screen; a 0 is a claim.
- Watch the bar at the top. It shows cost per acceptable outcome for each option as soon as enough lines are filled.
- Read the result at the end. It names the leading option, the break-even month for each, and the assumptions that decide the answer.
- Open the Reference example tab to see every line filled for a real-shaped case, and load it into your model to start from there.
Five rules that decide whether the number is evidence or a guess.
- Fill it from a trial, not from a guess. Escalation share, review minutes and the retry multiplier come from logs of real cases. A demo produces none of them. Nothing here is a benchmark: there is no industry number for your escalation rate.
- Argue about cost per acceptable outcome, not cost per case. Cost per case rewards a system that quietly declines the hard cases. Cost per acceptable outcome charges it for them.
- One definition of an acceptable outcome, one reviewer, all four options. The acceptable-outcome rate must come from one written definition, applied to every option by the same person. Otherwise the comparison is theatre.
- Blank means unknown. Zero means measured. A blank keeps the result off the screen until you know the number. Type a 0 only when you have measured a 0. "This system never retries" is a claim.
- Change the rows, keep the shape. Add a cost line your workload has and this model does not: data labelling, a vendor licence, a second reviewer for high-value decisions. The shape is the point: build kept apart from run, quality beside cost, and the operating lines visible.
Plain code that follows fixed rules. No model anywhere in the path.
The same code, rebuilt on the decision rules the agent build forced the team to write down.
A model drafts a recommendation. A person approves every one before it takes effect.
A model plans, calls tools and acts. People review a sample of cases and the exceptions.
Workload and manual baseline
What does this work cost today, done by hand?
Every option is measured against this. If the manual baseline is wrong, every break-even month below it is wrong by the same amount.
Such as INR or USD. Shown in front of every money figure. Not counted as a line.
One case is one unit of work: a purchase order, a ticket, a refund request. Take the count from last quarter, not from the plan.
The window the build has to pay for itself. Twelve is the usual budget horizon. Make it longer only if you can defend the longer horizon in the room.
How long one person takes to finish one case today. Time it on ten real cases.
Fully loaded: salary, benefits, seat. Use the rate of the people who actually do the work today.
Cases per month multiplied by months.
Total cases, times minutes per case, times the hourly rate. This is the number every option has to beat.
Team rates
What does an hour of each person cost?
Three rates, shared by all four options. Engineers build and handle escalations; reviewers approve output and finish declined cases.
What one engineer-day costs you. Used for the build and for upkeep.
Used for escalations: the cases an engineer has to look at.
Used for reviewing output and for finishing declined cases by hand. Often the same rate as the manual baseline.
Build cost, paid once
What does each option cost before it handles its first case?
Engineering days, converted at the day rate. Evaluation work is a separate line because it is the one most estimates skip, and it is where a model-based option earns the right to be trusted.
From the first line of code to a system that handles real cases. Include integration with the tools it calls.
An eval is a set of real cases with known-correct answers that the system is scored against. Building one is engineering time.
Build days plus eval days, times the engineering day rate.
Inference and tool calls, per month
What does the model bill actually include?
Inference is the cost of calling the model. Most business cases price the successful calls and stop there. The retry multiplier is where the rest goes.
How many times a case calls the model when everything works. An agent that plans, then acts, then checks its work makes several.
Total model calls divided by successful ones. Count timeouts, tool errors and loops that re-enter, not only refusals. 1.0 means the system never retries, which no real system does.
Average cost of one call, including the input tokens. Take it from a month of real invoices, not from the price list.
Lookups, searches, writes to other systems. Anything with a price per call or a rate limit.
Average across the tools it calls. Internal calls are not free: price the compute.
Cases per month, times calls per case, times the retry multiplier, times the cost per call.
Cases per month, times tool calls per case, times the cost per tool call.
Human in the loop, per month
How much human time does each option still need?
Three kinds of human time. Review is routine checking of output. Escalation is a case that needs an engineer. Declined is a case the system would not finish, which a person then does by hand. This section decides most comparisons.
100 for a model-assisted draft, because a person approves every one. A sample, such as 20, for a full agent.
Time it on real cases. Approving a good draft is quick; checking an agent that acted on its own is not.
Cases that stop and need somebody who can read the trace. Measure it in a trial. A demo will show you 0.
Reading the trace, working out what happened, fixing the case. Longer for an agent, because there is more trace to read.
Cases the system refuses or gives up on. They do not disappear: a person finishes them, at the manual rate.
Usually close to the manual baseline, because the person starts from nothing.
Cases per month, times the share reviewed, times minutes per review, at the reviewer rate.
Cases per month, times the share escalated, times minutes per escalation, at the engineer rate.
Cases per month, times the share declined, times manual minutes, at the reviewer rate.
Upkeep and operations, per month
What does it cost to keep the system trustworthy after launch?
A rules workflow needs almost none of this. A model-based option needs all of it, every month, and none of it appears in a demo.
Tracing, logging, the eval runner, the dashboards. Licence fees or the compute to run your own.
Engineering days a month spent adding cases to the eval set and fixing the ones that go stale.
Engineering days a YEAR spent re-running the eval set after a model or SDK update. Spread across the months by the model. Count every update the provider ships.
Engineering days a month spent checking that a prompt change did not break something that used to work.
Engineering days a month on incidents caused by the system. Take it from the on-call log.
Tooling, plus every engineering day above at the day rate, with the yearly line divided by twelve.
Quality
How often does each option produce an outcome you would accept?
This is what turns cost per case into cost per acceptable outcome. One written definition of "acceptable", applied to every option by the same reviewer.
The share of cases that meet the written definition with no rework. Measure it on the same set of cases for all four options.
Total cases, times the acceptable-outcome rate.
The outcome, read against the rubric.
Fill every one of the 83 lines to read the outcome. Until then this shows each total as soon as the lines that feed it are filled.
| Line | RulesPlain code. No model. | Rules v2Same code, rebuilt on the written spec. | AssistedModel drafts. A person approves each one. | AgentModel plans, calls tools, acts. |
|---|---|---|---|---|
| Run cost per monthModel, tools, review, escalation, declined cases and upkeep, added up. | — | — | — | — |
| Run cost for the periodRun cost per month, times months. | — | — | — | — |
| Total cost for the period, build plus runThe build, paid once, plus the run cost for the period. | — | — | — | — |
| Cost per caseTotal cost divided by total cases. The wrong number to argue about on its own. | — | — | — | — |
| Cost per acceptable outcomeTotal cost divided by acceptable outcomes. The number to argue about. | — | — | — | — |
| Net position against the manual baselineManual baseline minus total cost. Positive means the option saves money over the period. | — | — | — | — |
| Months to break even on the buildBuild cost divided by the monthly saving against doing it by hand. "Never" means the option costs more to run each month than the manual process. | — | — | — | — |
Check these assumptions first
The leading option first, then the full agent, then the rest. Each one is a line that decides the answer and is usually guessed.
An ordering agent across 40 sites, every line already filled.
A retailer in India raises about 2,000 purchase orders a month across 40 sites. Each one takes a buyer 25 minutes by hand.
The team built a full agent that reads the request, checks stock and supplier terms, and places the order. It worked. It also cost more to run each month than the buyers did.
Building it forced the team to write down every decision rule the buyers had been carrying in their heads. A second rules workflow, rebuilt on that written spec, took 35 engineering days and wins on cost per acceptable outcome.
Cost per case picks the original rules workflow. Cost per acceptable outcome picks the rebuilt one. That gap is the whole reason the second number exists.
Every figure is in Indian rupees at Indian rates. If you work elsewhere, change the four rates in sections 1 and 2 first. The shape of the comparison holds; the absolute numbers will not.
The agent build paid for itself by forcing the team to write the decision rules down. Ship the rebuilt rules workflow, and keep the specification under version control like any other code. Revisit the model when volume or the price of a call changes.
- Cost per case picks Rules workflow. Cost per acceptable outcome picks Rules v2, on the written spec. Argue about the second one: cost per case rewards a system that quietly declines the hard cases.
- Full agent with tools breaks even in month 17.7, after the 12-month period you modelled. It pays back, but only if you can defend the longer horizon.
| Line | RulesPlain code. No model. | Rules v2Same code, rebuilt on the written spec. | AssistedModel drafts. A person approves each one. | AgentModel plans, calls tools, acts. |
|---|---|---|---|---|
| Run cost per month | INR 50,467 | INR 37,733 | INR 1,42,418 | INR 3,67,967 |
| Run cost for the period | INR 6,05,600 | INR 4,52,800 | INR 17,09,016 | INR 44,15,600 |
| Total cost for the period, build plus run | INR 11,05,600 | INR 11,52,800 | INR 27,29,016 | INR 67,55,600 |
| Cost per case | INR 46.07 | INR 48.03 | INR 113.71 | INR 281.48 |
| Cost per acceptable outcome | INR 59.06 | INR 52.78 | INR 127.76 | INR 299.45 |
| Net position against the manual baseline | INR 48,94,400 | INR 48,47,200 | INR 32,70,984 | INR -7,55,600 |
| Months to break even on the build | 1.1 months | 1.5 months | 2.9 months | 17.7 months |
Check these assumptions first
- 01 Rules v2, on the written spec: escalations are 35% of run cost. That line decides the comparison. Measure it in a trial before you trust it.
- 02 Full agent with tools: upkeep is 50% of run cost. That line decides the comparison. Measure it in a trial before you trust it.
- 03 Rules workflow: declined cases are 38% of run cost. That line decides the comparison. Measure it in a trial before you trust it.
- 04 Model-assisted draft: upkeep is 41% of run cost. That line decides the comparison. Measure it in a trial before you trust it.
Workload and manual baseline
What does this work cost today, done by hand?
| Line | Unit | Value |
|---|---|---|
| Currency label | free text | INR |
| Cases per month | count | 2,000 |
| Months in the period | months | 12 |
| Minutes per case, by hand | minutes | 25 |
| Cost per hour of the person doing it by hand | INR per hour | 600 |
| Total cases in the period | worked out | 24,000 |
| Manual baseline cost for the period | worked out | INR 60,00,000 |
Team rates
What does an hour of each person cost?
| Line | Unit | Value |
|---|---|---|
| Engineering day rate | INR per day | 20,000 |
| Engineer cost per hour | INR per hour | 2,500 |
| Reviewer cost per hour | INR per hour | 600 |
Build cost, paid once
What does each option cost before it handles its first case?
| Line | Unit | RulesPlain code. No model. | Rules v2Same code, rebuilt on the written spec. | AssistedModel drafts. A person approves each one. | AgentModel plans, calls tools, acts. |
|---|---|---|---|---|---|
| Engineering days to build | days | 25 | 35 | 45 | 95 |
| Engineering days for evals and a test set | days | 0 | 0 | 6 | 22 |
| Total build cost | worked out | INR 5,00,000 | INR 7,00,000 | INR 10,20,000 | INR 23,40,000 |
Inference and tool calls, per month
What does the model bill actually include?
| Line | Unit | RulesPlain code. No model. | Rules v2Same code, rebuilt on the written spec. | AssistedModel drafts. A person approves each one. | AgentModel plans, calls tools, acts. |
|---|---|---|---|---|---|
| Model calls per case, successful | calls per case | 0 | 0 | 1.2 | 6 |
| Retry and failure multiplieroften left out | multiplier, 1.0 = no retries | 1 | 1 | 1.15 | 1.35 |
| Cost per model call | INR | 0 | 0 | 1.8 | 2.5 |
| Tool and API calls per case | calls per case | 0.2 | 0.2 | 1 | 5 |
| Cost per tool call | INR | 0.5 | 0.5 | 0.5 | 0.5 |
| Model cost per month | worked out | INR 0 | INR 0 | INR 4,968 | INR 40,500 |
| Tool cost per month | worked out | INR 200 | INR 200 | INR 1,000 | INR 5,000 |
Human in the loop, per month
How much human time does each option still need?
| Line | Unit | RulesPlain code. No model. | Rules v2Same code, rebuilt on the written spec. | AssistedModel drafts. A person approves each one. | AgentModel plans, calls tools, acts. |
|---|---|---|---|---|---|
| Share of cases a person reviews | % | 12 | 8 | 100 | 20 |
| Review minutes per reviewed caseoften left out | minutes | 6 | 6 | 2 | 4 |
| Share of cases escalated to an engineer | % | 1 | 0.8 | 1.5 | 4 |
| Engineer minutes per escalationoften left out | minutes | 20 | 20 | 25 | 35 |
| Share of cases declined or abandonedoften left out | % | 8 | 4 | 3 | 2 |
| Manual minutes per declined case | minutes | 12 | 12 | 12 | 12 |
| Review cost per month | worked out | INR 14,400 | INR 9,600 | INR 40,000 | INR 16,000 |
| Escalation cost per month | worked out | INR 16,667 | INR 13,333 | INR 31,250 | INR 1,16,667 |
| Declined-case cost per month | worked out | INR 19,200 | INR 9,600 | INR 7,200 | INR 4,800 |
Upkeep and operations, per month
What does it cost to keep the system trustworthy after launch?
| Line | Unit | RulesPlain code. No model. | Rules v2Same code, rebuilt on the written spec. | AssistedModel drafts. A person approves each one. | AgentModel plans, calls tools, acts. |
|---|---|---|---|---|---|
| Observability and eval toolingoften left out | INR per month | 0 | 0 | 8,000 | 35,000 |
| Eval maintenanceoften left out | engineer days per month | 0 | 0 | 1 | 3 |
| Model-version re-qualificationoften left out | engineer days per year | 0 | 0 | 9 | 24 |
| Prompt and regression testingoften left out | engineer days per month | 0 | 0.25 | 0.5 | 1.5 |
| Incident and on-call loadoften left out | engineer days per month | 0 | 0 | 0.25 | 1 |
| Upkeep cost per month | worked out | INR 0 | INR 5,000 | INR 58,000 | INR 1,85,000 |
Quality
How often does each option produce an outcome you would accept?
| Line | Unit | RulesPlain code. No model. | Rules v2Same code, rebuilt on the written spec. | AssistedModel drafts. A person approves each one. | AgentModel plans, calls tools, acts. |
|---|---|---|---|---|---|
| Acceptable-outcome rate | % | 78 | 91 | 89 | 94 |
| Acceptable outcomes in the period | worked out | 18,720 | 21,840 | 21,360 | 22,560 |
Agree this before the numbers go in, not after.
The leading option is the one with the lowest cost per acceptable outcome among the options that save money against the manual baseline over the period. The row that matches your result is highlighted.
Three flags sit beside the outcome and do not change it. Disagreement: cost per case and cost per acceptable outcome pick different options. Never breaks even: an option costs more to run each month than doing it by hand. Beyond the period: an option pays back, but after the months you modelled.
The model bill is usually the smallest line.
Each of these is marked often left out in the model above. Together they are most of what it costs to run an agent, and none of them appears in a demo.
Retry and failure multiplier
Total model calls divided by successful ones. Count timeouts, tool errors and loops that re-enter, not only refusals. 1.0 means the system never retries, which no real system does.
Review minutes per reviewed case
Time it on real cases. Approving a good draft is quick; checking an agent that acted on its own is not.
Engineer minutes per escalation
Reading the trace, working out what happened, fixing the case. Longer for an agent, because there is more trace to read.
Share of cases declined or abandoned
Cases the system refuses or gives up on. They do not disappear: a person finishes them, at the manual rate.
Observability and eval tooling
Tracing, logging, the eval runner, the dashboards. Licence fees or the compute to run your own.
Eval maintenance
Engineering days a month spent adding cases to the eval set and fixing the ones that go stale.
Model-version re-qualification
Engineering days a YEAR spent re-running the eval set after a model or SDK update. Spread across the months by the model. Count every update the provider ships.
Prompt and regression testing
Engineering days a month spent checking that a prompt change did not break something that used to work.
Incident and on-call load
Engineering days a month on incidents caused by the system. Take it from the on-call log.
A number you can take to a budget conversation. Not the whole argument.
- It assumes steady volume, one workload and stable rates for the whole period.
- It does not price revenue upside. A system that wins new business is not captured by a cost line.
- It does not price the risk of a wrong action that cannot be undone. That belongs in the Agent Authority Review, not in a cell here.
- It does not price the option value of the capability you build. If that is the argument, write it in the decision record in words, not as a number in this sheet.
An agent proposal is also a commitment to operate an agent. This tool turns that sentence into a number. Sometimes the number says build it. The reference example is a case where it did not, and where the build was still worth doing, because it produced the written specification that made the cheap option good.
See the cohort → All resources → The same model as an Excel workbook ↓
Published 14 September 2026 · free to use and to pass on