Before approving the next investment in a successful AI pilot, separate what the system demonstrated from what the organisation has agreed to operate. A technical result can justify further testing without justifying a production commitment.
Research hosted by Stanford’s SCALE Initiative illustrates the distinction: in two anonymized operational cases from one large public education system, technically viable AI systems could not advance to broader rollout for institutional rather than technical reasons. Those cases do not establish an enterprise-wide failure pattern. They do give technology owners a reason to examine readiness beyond model capability.
A demo validates a bounded capability
State precisely what the pilot demonstrated, under which inputs, system boundaries and evaluation criteria. If it did not exercise production permissions, integration failures or operational handoffs, keep those questions open. Do not let approval of the technical result implicitly approve an untested operating model.
Challenges in Deploying Machine Learning: A Survey of Case Studies reviewed published deployment reports across use cases, industries and applications. It found practitioner issues at every stage of deployment. This concerns machine learning broadly, not generative-AI pilots specifically, and the finding establishes neither the frequency nor the relative importance of those issues.
Use that finding to widen the assessment, not to estimate a failure rate. For the next budget decision, ask the team to separate demonstrated results, unresolved dependencies and assumptions that would invalidate the proposed deployment. AI production readiness should be assessed against a defined use case, not inferred from a successful demonstration.
Condition one: fit the operating workflow
The Stanford public-systems source identifies five readiness areas: institutional and operational compatibility, data ecosystem maturity, human oversight capacity, fiscal sustainability and regulatory alignment readiness. The three conditions used here group those areas into a practical review. They are editorial guidance, not a validated formula for production success.
Begin with the receiving workflow. Specify when the AI may act, what information it receives and who takes responsibility for its output. Distinguish a recommendation from an action that changes a system of record. Make both the integration boundary and the authority boundary explicit.
Define eligible inputs, permissions and required workflow state.
Bound the output and authority to act.
Name the receiving person or system and acceptance conditions.
Specify escalation, recovery and responsibility for closure.
Suggested review sequence. Test exceptions wherever they can arise.
Trace the proposed operating path and assign responsibility at each step.Test departures from the intended path before expanding scope. Decide what should happen when required data is unavailable, a downstream system rejects an action or a reviewer declines the output. Agree rules for pausing, retrying, reversing or routing work before approving the integration.
Do not assume adoption includes workflow redesign. In McKinsey’s global survey published in March 2025, 21% of respondents reporting organisational gen-AI use said their organisations had fundamentally redesigned at least some workflows. This was self-reported; the available passage does not provide sample size, sampling frame or geographic breakdown. It does not show that redesign causes production success. For this review, ask which changes the proposed use case requires rather than treating the percentage as a benchmark.
Condition two: make data and judgment accountable
Data ecosystem maturity and human oversight capacity are separate areas in the public-systems framework. Review them together, while keeping their ownership distinct: who is responsible for the information entering the system, and who has authority over the resulting decision?
For each production input, identify who owns quality, access, refresh and correction. Define how missing, stale or conflicting information should affect system behaviour. Set traceability requirements so an authorised reviewer can reconstruct the relevant input, output and subsequent decision, within applicable access and retention constraints.
Treat “human review” as an operating capability to test. Name the reviewer, the evidence they receive, their authority to reject an output and the escalation owner. Keep final decision authority explicit. A review requirement without an accountable decision-maker should remain an unresolved dependency.
Test expected review demand against funded capacity. Decide in advance whether work should wait, follow an approved alternative or fall outside the AI’s permitted scope when that capacity is unavailable. Set use-case-specific acceptance criteria rather than treating the presence of a reviewer as sufficient assurance.
Condition three: sustain the operating commitment
Fiscal sustainability and regulatory alignment readiness complete the five areas identified in the public-systems source. Carry both into the ongoing operating review, rather than treating them as one-time launch approvals.
Build the cost assessment around the proposed workflow. Include model and infrastructure spend alongside integration maintenance, evaluation, human review, support and governance work where applicable. Make workload and exception assumptions visible, then specify which changes would trigger a new funding decision or a narrower deployment.
Assign responsibility for identifying applicable regulatory obligations and the evidence needed to demonstrate compliance. Name who approves changes to the model, data access, decision authority and intended use. Require a suspension and recovery route for circumstances in which the agreed operating conditions no longer hold.
Before handover, ask the production owner to accept the maintenance scope, budget assumptions and change controls explicitly. If operation still depends on uncommitted support from the pilot team, record that as an unresolved dependency in the approval decision.
Make the next decision smaller and explicit
Use the readiness review to authorise a defined commitment. Record its scope, supporting evidence, unresolved assumptions, accountable owners and conditions for reconsideration. For each unresolved assumption, specify the test needed and who will decide whether its result is acceptable. The evidence discussed here supplies no universal readiness thresholds.
Change workflow, integration or oversight arrangements, then test again.
Limit inputs and authority; assign owners, monitoring and stop conditions.
End the proposed deployment rather than accept unresolved obligations.
Suggested decision criteria, not predictions of production success.
Match the next commitment to the conditions the organisation can support.A named owner and a test plan justify resolving an uncertainty, not assuming it away. Advance only the scope whose required operating conditions have been demonstrated and accepted. Constraining or stopping that scope does not erase the technical achievement. It keeps the production commitment within what the organisation can operate and defend.
Written by the studio, from work in delivery
