Before approving an AI feature on a token estimate, ask what the estimate allows for when an output is rejected: another model call, more context, human correction, or abandonment. Require those paths to be accounted for before treating the estimate as an operating budget.
Build the AI feature cost model around one bounded workflow, with separate accounts for LLM use, human intervention and infrastructure. Mark what is assumed and what is measured. The purpose is to make a defensible commitment about what the team can operate, not to produce a universal forecast.
Count the workflow, not just the call
ACEM, a research preprint proposing a cost-estimation model for agentic software engineering, separates LLM, human-in-the-loop and infrastructure costs. It explicitly represents token overhead from rejected outputs and retries, alongside token growth as context accumulates. That provides a structure to consider, not a validated estimator for every AI feature.
Human work also warrants a wider boundary than final approval. Qualitative research drawing on 15 interviews and workflow artifacts in two early-adopting German technology firms describes intervention during drafting, refinement and review. Oversight was distributed through the studied workflows. The study does not quantify review effort or labour cost, or establish a universal workflow pattern.
Use these perspectives to inspect the whole path from input to disposition. Define what counts as an accepted outcome, but include rejected and abandoned work in the accounting boundary. If you report cost per accepted outcome, state how unsuccessful attempts are allocated. Otherwise, the denominator excludes work the team still has to fund.
Set the boundaries before calculating
Start with a boundary statement: inputs, accepted outputs, rejection conditions, review points, infrastructure dependencies and accountable owner. Specify the reporting period, shared-cost allocation and exclusions. Keep implementation effort separate from recurring operation, and state where maintenance and governance work belong.
Capture usage across initial attempts, retries and revisions, including changing context.
Capture intervention, review and escalation work by role and workflow stage.
Identify supporting services and allocate dedicated and shared costs.
The dimensions come from ACEM's proposed agentic-development model. The measurement guidance is an editorial recommendation.
Maintain three separate cost accounts, using the following measurement recommendations.Make retry and revision behaviour explicit. Distinguish a technical retry from a revision prompted by an unacceptable output, then define each trigger, stopping condition and escalation path. Track context size separately rather than assuming repeated calls receive identical inputs.
Treat these behaviours as explanations of usage, not automatic surcharges. If observed token consumption already includes retries, adding a retry allowance would count that consumption twice.
Measure retries and context locally
ACEM leaves its constants symbolic pending empirical grounding and calls for calibration and testing with real project data. Its predictive accuracy is therefore not established by the supplied evidence. This limitation concerns ACEM, not every cost model.
Before deployment measurements are available, label retry frequency, revision depth and context growth as assumptions. Give each an owner and a measurement plan. Use alternative scenarios to test whether uncertainty changes the proposed commitment, but do not present them as observed distributions or forecasts.
For OpenAI API deployments, official documentation identifies request IDs and rate-limit headers covering request and token limits, remaining capacity and resets. It recommends logging request IDs in production for troubleshooting. These provider-specific signals do not supply retry counts, token prices or human labour costs; rate-limit capacity is not a record of actual consumption.
Record uncertain inputs, owners and affected decisions.
Link workflow runs, model attempts and human interventions.
Capture usage, context size, retry reasons and outcomes.
Apply verified rates and allocations; identify missing data.
Replace assumptions and revisit the operating commitment.
Recommended measurement sequence, not a validated forecasting method.
Connect assumptions to observations at the workflow level before revising the estimate.At the application layer, record the workflow identifier, attempt sequence, model and configuration, reported token usage, reason for repetition and final disposition. Measure context size without making raw prompt retention a default requirement. Obtain applicable provider rates separately and record their effective dates. Retain the observation period and configuration with each estimate so its scope remains clear.
Put human review where the work happens
Use drafting, refinement and review as inspection points, not a mandatory workflow template. The qualitative study supports looking beyond a single release gate in its setting; it supplies no staffing or review-time benchmark.
- — For each human intervention, record:
- — Trigger: why the person became involved.
- — Work: active time spent assessing, correcting or deciding.
- — Authority: who can revise, approve, reject or escalate.
- — Outcome: acceptance, another attempt, escalation or abandonment.
- — Cost basis: the agreed internal labour rate and allocation method.
Separate active review time from time waiting for a reviewer. Track both for operating decisions, but do not price the entire elapsed interval as active labour. Link interventions to the same workflow record as model attempts so correction before final sign-off remains visible.
Before removing a review step, identify the judgment it requires, the authority it carries and who would inherit responsibility. Evaluate approval-boundary changes with the risk owner. A lower handling-time estimate alone is not a sufficient basis for removing oversight.
Use the model to make an operating decision
Prioritise the uncertainties that could change feature scope, architecture or approval to scale. For each unresolved input, ask whether the commitment still holds under the scenarios you have explicitly chosen to examine. Spend measurement effort where the answer is uncertain.
Agree inputs, exception paths and stopping conditions.
Assess usage, revisions and review against agreed quality requirements.
Evaluate retry limits and review changes with the risk owner.
Assign measurements that could change the commitment.
Assign exceptions, cost reconciliation and estimate updates.
Recommended decision criteria, not empirical findings or rankings.
Use these checks to turn the cost model into decisions and assigned work.Make approval conditional on a bounded cost account, a measurement plan and named owners for unresolved assumptions. Agree the quality, cost and risk conditions that would trigger a pause or redesign. If an unmeasured input could reverse the scaling decision, keep deployment bounded until that uncertainty is resolved.
Written by the studio, from work in delivery
