Why AI Pilots Stall, and How to Get Them to Production

A stalled pilot is a familiar sight: a demo that impressed leadership, a Slack channel that went quiet, and a vague plan to "revisit next quarter". The model usually worked fine. What failed was everything around it.

The numbers say this is the norm rather than the exception. An MIT NANDA report from 2025, as summarised by Virtualization Review, found that only 5% of custom enterprise AI tools reach production: 60% of organisations evaluated such systems, 20% reached a pilot and 5% reached production. Treat those figures with care, as the sample was limited and outcomes self-reported, but the direction matches what analysts predicted. Gartner said in 2024 that at least 30% of generative AI projects would be abandoned after proof of concept by the end of 2025, and in 2025 that more than 40% of agentic AI projects would be cancelled by the end of 2027, citing "escalating costs and unclear business value".

The five reasons AI pilots stall

The five blockers below are the ones I check first when a pilot has stalled, and they are almost all decisions rather than code.

Blocker 1: nobody owns the result

A pilot without a business owner has nobody to say "this is good enough, ship it", so it never ships. IT teams and vendors can build, but they cannot decide that the claims team will change how it works. That decision belongs to the person whose numbers improve if the pilot succeeds.

The fix is to name that person before the build starts and tie the pilot to a metric they already care about: minutes per ticket, invoices processed per day, time to first response. If no one will put their name to it, that is useful information: the pilot is solving a problem nobody has.

Blocker 2: no way to measure quality

Without an evaluation set, every review becomes an argument about anecdotes, and the safest decision is always to wait. Someone finds one bad answer, confidence drops, and the pilot goes back for "more testing" with no definition of done.

The fix is cheap: collect a few dozen to a few hundred real examples with the expected output, agree a pass mark with the owner, and run every change against them. Then the conversation changes from "I saw a bad answer" to something like "accuracy on the test set went up this week, and the remaining failures fall into three types". I go deeper on this in the evals that stop AI agent projects failing.

Blocker 3: the pilot never touches real data

Pilots built on a one-off export work beautifully and then break on contact with production, where data is messier, permissions are stricter and the format changes. Getting real access is often slow because security reviews are slow, so teams skip it to keep momentum and pay for it later.

The fix is to request read-only access to the real systems on day one, in parallel with discovery. If access takes three weeks, that is the first finding of the pilot, and it is better to learn it in week one than in week eight.

Blocker 4: the workflow around the AI does not change

An AI tool dropped into an unchanged process becomes one more tab people forget to open. The MIT NANDA findings point the same way: the report blamed failures on fragile workflows and tools that do not fit day-to-day operations, and said that "most GenAI systems do not retain feedback, adapt to context, or improve over time".

The fix is to design the work, not just the model call. Decide who sees the AI's output, where it appears in the tools they already use, what they do when it is wrong, and how their corrections feed back into the system. Then train the people involved, briefly and in their own tools. Checking AI output is becoming part of every role that works alongside an agent.

Blocker 5: nobody budgeted for running costs

A pilot is funded as a project, but production is an ongoing cost: model usage, hosting, monitoring and someone to fix things. When that line is missing from next year's budget, the safest choice for everyone is to leave the pilot where it is.

The fix is to estimate cost per task during the pilot, multiply by expected volume, and agree who pays before rollout. Running costs can usually be cut significantly with caching, batching and smaller models for simple steps, but only if someone measures them first. For how a pilot budget breaks down, see what an FDE pilot costs; for the return side, how to measure AI automation ROI.

How to rescue a pilot that has already stalled

A stalled pilot can usually be restarted in a few weeks by fixing the blockers in order rather than rebuilding the model. When I look at a stalled pilot, I work through it in this order:

• Find the owner, or confirm there is none and stop the pilot honestly.
• Build the evaluation set from real cases and measure where the current system actually stands.
• Move it onto real data in the real environment, with proper access controls.
• Redesign the workflow with the people who will use it, then roll out to a small group.
• Put a cost per task on it, from a usage ledger of real model calls, and get the running budget agreed.

Often the existing code is mostly fine. What changes is that the pilot now has a decision-maker, a measure of quality and a path into daily work. This is the core of what a forward deployed engineer does; my buyer's guide to hiring a freelance FDE covers how to bring one in.

Work with me

I rescue stalled AI pilots and take new ones to production as a freelance forward deployed engineer. See AI agent development and AI integration, browse my work, and book a call to walk through where your pilot is stuck.