What does an AI workflow cost to run?
The useful number is cost per accepted task: everything you spend to produce results a reviewer accepts, divided by the number of results accepted. Token prices alone hide retries, tool calls, and review time. This guide gives the formula, the five cost lines, an illustrative example, and the rules for comparing two options fairly.
The formula
Cost per accepted task = (model calls + tools and integrations + infrastructure + retries and corrections + review labor) / accepted tasks
Two choices make the number honest. The numerator includes everything, not just the provider invoice. The denominator counts only results a reviewer accepted, not attempts. A workflow that produces 100 drafts of which 60 are usable costs whatever it costs divided by 60.
The five cost lines
- Model calls
- Tokens in and out, across every step, including classification, retrieval prompts, drafting, and self-checks. Multi-step work multiplies this line. Gartner puts agentic models at five to thirty times the tokens per task of a standard chatbot.
- Tools and integrations
- Per-call charges for search, document processing, enrichment, and integration platforms. Often small per task, easy to forget, and priced in different units.
- Infrastructure
- Hosting, storage, queues, vector stores, monitoring. Allocate a share to the workflow rather than leaving it in a general bill.
- Retries and corrections
- Every rerun after a timeout, a failed validation, or a rejected output costs model and tool charges again. Count them; they are where “cheap” models get expensive.
- Review labor
- The minutes a person spends approving, editing, or rejecting, at a loaded hourly rate. Usually the largest line in a new workflow and the one that falls as quality improves.
An illustrative example
The numbers below are invented to show the arithmetic. They are not a measurement, a benchmark, or a quote. A baseline workflow and a candidate with a cheaper model handle the same 100 tasks in a month.
| Cost line | Baseline | Candidate |
|---|---|---|
| Model charges, including retries | $60 | $40 |
| Tools and hosting, allocated | $20 | $20 |
| Review and correction labor | $120 | $180 |
| Total operating cost | $200 | $240 |
| Accepted tasks | 80 of 100 | 75 of 100 |
| Cost per accepted task | $2.50 | $3.20 |
The candidate spends a third less on model calls and costs 28% more per accepted task, because reviewers spend longer correcting it and accept fewer results. Judged on the provider invoice, it looks like a saving. Judged on accepted work, it is not. The cost calculator runs this arithmetic on your own numbers in the browser; nothing is sent anywhere.
Compare at the same quality floor
A comparison is only fair when both options face the same work and the same standard.
- Same cases. Run both options on the same test set, including the awkward cases.
- Same acceptance criteria. Written before the run, applied by the same reviewer.
- Same period and volume. Fixed costs allocated the same way.
- Count everything. Retries, tool calls, and review minutes, not just the model line.
- Keep failures visible. A candidate that is cheaper on average but fails a sensitive case has not passed. Record that case separately.
Why agent workflows cost more than a chatbot
A chatbot answers once. An agent classifies, retrieves, drafts, checks its own work, and may call several tools, each step with its own prompt and context. That is why Gartner’s five-to-thirty-times figure exists, and why the cost line must be watched per task. It is also why the cheapest way to lower cost is often not a cheaper model but a shorter path: better inputs, fewer retries, and less context per step.
Ways to lower cost without lowering quality
- Route simple cases to a smaller model and reserve the larger model for the hard ones.
- Cache repeated context (policies, templates) instead of resending it on every call.
- Send less: retrieve the relevant passage, not the whole document.
- Fix inputs upstream so retries fall: cleaner intake forms, complete records, readable scans.
- Batch overnight work where latency does not matter.
- Move reversible, low-risk items from full review to sample review once the override rate is low.
Each change is a candidate to test against the baseline with the rules above, not a setting to flip. The evaluation and optimization service is built around that comparison.
What to record each month
Accepted tasks, total operating cost by line, cost per accepted task, override rate, escalation rate, and any sensitive failure. Six numbers, one page, same format every month. Trends matter more than any single reading.
Sources
External references used in this guide. Figures are quoted with their original baseline; none is a measurement of our own work.