Why AI pilots fail to scale: a process problem, not a technology problem
AI pilots fail to scale because organisations try to scale the tool before they have defined the work. A pilot succeeds inside one team, with one person’s judgement filling the gaps. At scale, an agent needs a shared, governed and current way of working to measure itself against – and most organisations have never written that down.
The research points the same way. MIT’s NANDA initiative found that around 95% of enterprise generative AI pilots stall, delivering little to no measurable impact on the P&L. The cause, it argues, is not the quality of the models but a “learning gap”: generic tools don’t learn from or adapt to the organisation’s workflows (MIT NANDA, The GenAI Divide, 2025, via Fortune).
This article draws on the opening episode of Ground Truth, Gluu’s video series on building the process foundation for AI at scale, in which Søren Pommer talks with Michael Hansen, Gluu’s Head of AI.
Key takeaways
- Individual AI wins are real, but they don’t add up to organisational capability. The unit of change is the way of working, not the individual.
- Agents need three things: a prompt (how you ask), context (what to know) and intent (what to optimise for, and when to stop). Most organisations are missing intent.
- Better models don’t close the gap. Your trade-offs, exceptions and definition of a good outcome are not in any training data.
- Before an agent acts inside a process, you need a governed and current process, explicit boundaries and an audit trail.
- Start with one or two key processes, not a documentation programme.
Watch the full conversation with Michael Hansen from Gluu’s Ground Truth series. The written version below covers the main points.
Why do AI pilots fail to scale?
Because a pilot and a scaled deployment answer different questions. A pilot asks: can this agent do this task well? Scaling asks: does this agent make the whole organisation work better?
“The organizations that scale AI, they’re not the ones that have the best model or the best individual agent. They’re the ones that know how the organization works and how the agent contributes to success.”
Michael Hansen, Head of AI, Data & Analytics at Gluu
That is why so many companies see plenty of AI activity and little change. Everyone has an account and some people go home earlier, but, as Søren puts it, everybody is still running off in their own direction. “They’re just running faster.” Scaling also brings back the familiar problems of integration, common definitions and common data. The old rule still applies: automate a mess and you get an automated mess.
| Treated as a technology problem | Treated as a process problem | |
|---|---|---|
| Unit of change | The individual user | The way of working |
| Success measure | Logins, prompts, licences | Outcomes of the end-to-end process |
| Where knowledge lives | In people’s heads and chat threads | In a shared, governed way of working |
| What the agent optimises for | The task it was given | The outcome the organisation wants |
| What scaling exposes | Integration gaps and conflicting definitions | Common definitions people and agents share |
What is missing: a defined way of working
In AI, ground truth is the verified data you check a model against. The organisational equivalent, in Michael’s words, is “a governed current way of working” that people and agents can measure their work against. It is not just a question of what the organisation does, but “how we’re doing this and who says that we should do it in this specific way.”
Very few organisations have that complete picture. The knowledge is spread across documents and systems and, as Michael puts it, “just basically in people’s heads”, and it has never been codified. At pilot scale, a person quietly fills the gaps. At organisational scale, an agent can’t. What it finds in unstructured data is often anecdotal evidence of how certain people happen to do things, not how the organisation wants to work.
“Our processes have been anchored from top to bottom in the group”. Read case
Camilla Larsen, Sr. Process Manager, Norlys
“it’s really hard to optimize work that is fundamentally different if you’ve never defined how it worked in the first place.”
Michael Hansen, Head of AI, Data & Analytics at Gluu
A process map alone doesn’t fix this. As Søren puts it, people don’t execute diagrams, they execute ways of working: the map plus the conversation around it, the assets and the work instructions that together describe best practice.
Context tells the agent what to know. Intent tells it what to want
The way we instruct AI has moved through three stages. Intent engineering is a term from AI commentator Nate B. Jones, which Michael first heard on Jones’s podcast. He defines it as “making it explicit what an agent should optimize for and when it should stop” (see what intent engineering means for AI leaders).
| Stage | The question it answers | Why it matters now |
|---|---|---|
| Prompt engineering | How do I talk to the model? | Less important as models improve |
| Context engineering | What does the agent need to know? | Still essential |
| Intent engineering | What should it optimise for – and when should it stop? | Increasingly the missing piece |
Context describes the environment an agent works in. Intent carries the organisation’s decisions about what it wants to achieve, so the agent doesn’t work against the overall direction the company wants to take. That intent is not the strategy deck. In Gluu’s view it belongs in a living process system, enriched with systems, work instructions and day-to-day dialogue, that represents how the organisation wants to operate.
“Having a disconnect between what the organization wants and what the agent does is quite dangerous from an overall perspective.”
Michael Hansen, Head of AI, Data & Analytics at Gluu
Why won’t better models close the gap?
Models get better every month, so why not wait? Because the missing piece isn’t intelligence. Michael agrees the models are getting smarter, but says “the intent is becoming more and more relevant”. An agent “wouldn’t know anything about the trade-offs” an organisation makes internally, says Michael, or about its exceptions to rules and processes, or what a good outcome is in relation to its strategy. None of that is “baked into the learning, the training data” these models are based on.
Michael frames it with the classic capability model: people, process and technology, a three-legged stool. Technology is one leg. It keeps getting stronger, but what a company does still depends on its processes and its people, with the data aspects in place too.
The risk isn’t only stalled pilots. A failing agent gets noticed and fixed. An agent that “seemingly works perfectly, but it’s not aligned with what actual success looks like” is, in Michael’s words, “extremely dangerous”. Søren adds the distinction between reversible and irreversible outcomes: an internal failure can be fixed, but an agent sending the wrong invoices to every customer is close to irreversible.
What does it look like when you define the outcome?
Where Michael sees AI go wrong, it is usually because goals are set one-dimensionally, around a single metric. You get what you measure, and AI will optimise for that one number at the expense of everything else. Yet AI doesn’t need a single focus: if you are aware of the outcomes that matter and define them, it can optimise for more than one – “you can have your cake and eat it”, as Michael puts it. His example is Gluu’s own support: resolve customer cases faster while keeping customers satisfied.
Support is also where Gluu has gone furthest itself, moving from a rather manual process to one driven much more by AI. According to Michael, it has improved both the speed with which cases are handled and the quality of the results.
What has to be true before an agent acts inside a process?
There is a line between AI that assists a person and AI that operates on its own. Michael names three conditions before you cross it:
| Condition | What it means in practice |
|---|---|
| A governed, current process | An up-to-date, governed definition of how the work is done, with an accountable process owner – the ground truth for the agent |
| Explicit boundaries | Where the agent works, where people work, and which systems it may touch |
| An audit trail | A person or an external authority can verify that the process was followed by every actor in it |
Accountability comes with it: someone must own the outcome and make sure the agent fits into the right process and works as intended. And anything irreversible, whether towards a customer, a colleague or a system, is something Michael would not leave to an agent on its own: “I would not just let it go.”
Where should you start?
Not with a documentation programme. “The goal is never documentation for the sake of documentation,” says Michael. Pick one or two key processes, make them explicit and work from there, building over time towards “one source of truth that people and also agents can read and understand”. Governed, versioned processes help keep that source current once agents depend on it. Or, as Søren sums it up: first the process, then stability, then AI.
For the broader picture, see how processes make AI work across an organisation and why process initiatives stall in year one.
If you want to try this on one process, you can start a free trial of Gluu below.
Gluu free 30-day trial. No credit card required. Start from €24 / year.
More from Ground Truth Season 1: why AI usage is not the same as change (with Irina Holt Christensen), where an organisation’s intent actually lives (with Etienne Venter) and why AI in ERP projects makes unclarity faster (with Henrik Hasselbalch).
FAQ – why AI pilots fail to scale
Because the pilot relied on people to fill gaps that nobody has defined. At scale, agents need a shared, governed and current way of working – including what good outcomes look like – and most organisations have never codified it.
Mostly not. Models are improving fast, but an organisation’s trade-offs, exceptions and definition of success are not in any training data. Technology is one leg of the people-process-technology stool.
Context tells an agent what it needs to know about its environment. Intent tells it what to optimise for and when to stop, so it works towards what the organisation actually wants.
A governed and current process, explicit boundaries between agent and human work, and an audit trail that lets someone verify the process was followed.
With one or two key processes. Make them explicit, define the outcomes that matter, and expand from there towards a single source of truth for people and agents.
