We bring this topic up because of a pattern we keep running into when we’re asked to look at an AI initiative that has stalled. The post-mortem almost always starts with the technology. The model hallucinated, the vendor overpromised, or the accuracy was never there. We understand why those explanations are comfortable. They put the problem outside the building, and the fix sounds simple: wait for the next model, or switch vendors.
In the reviews we run, though, that explanation rarely makes it past the first hour. When you trace a stalled program back far enough, you tend to find the same short list of causes, and the model usually isn’t on it. Nobody owned the outcome, the process was broken before anyone automated it, the data wasn’t ready to be exposed, or nobody agreed what success meant. In most cases the model did roughly what it was built to do, and it was the organization around it that wasn’t ready.
This is part three of our series on AI with discipline. Part one argued that the gap between using AI and getting paid for it is an operating problem, and part two covered measuring value as work absorbed. In this post we want to take the four failure modes one at a time, each with a public case, and finish with a checklist to run before your next launch.
Sources: BC Civil Resolution Tribunal, February 2024; CNBC, June 2024; The Markup, March 2024; University of Texas System audit, 2016.
Failure One: No Owner for the Outcome
Air Canada, February 2024. A customer booking travel after a death in the family asked the airline’s website chatbot about bereavement fares. The chatbot told him he could claim the discount after flying, but the airline’s actual policy didn’t allow that. When he was refused, Air Canada argued before British Columbia’s Civil Resolution Tribunal that the chatbot was a separate legal entity responsible for its own actions. On February 14, 2024, the tribunal rejected that argument and ordered the airline to pay about CA$812 in damages, interest and fees.
The dollar amount was small, but we think the lesson is a big one. When an AI speaks for your organization, customers are going to treat it as your organization speaking, and so will a tribunal. Someone has to own what it says, the same way a department head owns what their staff tell customers. That means deciding which questions it can answer, which source it answers from, how errors get caught, and who fixes them.
In the stalled programs we see, ownership usually sits with IT, which owns the platform, or with a vendor, which owns the model. Neither one owns the business outcome. The fix we recommend is to name one accountable business owner per use case before anything gets built, meaning someone whose name is on the result and who has the authority to change the process around it. The business should drive the use case, because the people who do the work know where it goes wrong. IT’s job doesn’t shrink. It owns the guardrails: security, compliance, governance and auditing, including the record of what the AI said and when.
Failure Two: Automating a Broken Process
McDonald’s and IBM, June 2024. McDonald’s started testing IBM’s automated voice ordering in 2021 and ran it in more than 100 U.S. drive-thrus. In June 2024, CNBC reported that the company was ending the test, with the technology switched off in those restaurants by July 26. Widely shared videos had shown orders going wrong. McDonald’s said it still saw a future for voice ordering.
To be fair, a drive-thru order is a hard process to hand to software. There’s background noise, several voices in one car, and customers changing their minds mid-sentence. With that said, the lesson applies well beyond fast food. AI applied to a messy process inherits the mess. If the steps are unclear, the inputs are inconsistent and the exceptions live in people’s heads, the AI reproduces all of it, quickly and in front of customers.
We’ve found that the organizations that avoid this write the process down first: the steps, the inputs, the exceptions and who handles each one. That exercise often shows the process can be simplified before any AI is involved, and sometimes the simplification delivers most of the value on its own. It also shows which layer of the platform the problem actually needs, whether that’s plain workflow automation, an assistant answering from approved content, or a custom agent, which is where most of the money is won or lost. We see that more often than you’d think. Part five of this series is a filter for exactly that decision.
Failure Three: Data That Isn’t Ready to Be Exposed
New York City’s MyCity chatbot, March 2024. The city launched MyCity in October 2023 to help small-business owners work through city rules. On March 29, 2024, an investigation by The Markup and THE CITY reported that the chatbot was telling business owners they could do things that are illegal in New York, including taking a cut of workers’ tips and refusing tenants who use housing vouchers. The city kept the tool online, added disclaimers and kept refining it.
A chatbot that answers regulatory questions is only as good as the authoritative content behind it and the testing done before the public sees its answers. In a regulated area, a wrong answer is more than embarrassing, because it’s advice someone may actually act on.
Inside most organizations, the same problem shows up more quietly, and it’s the part we’d check first. Assistants like Microsoft 365 Copilot answer from whatever the user can already reach, which includes every overshared site, stale permission and outdated policy draft. Data that was harmless while nobody could find it becomes a real risk once AI can find it for them. When we talk about data readiness, we mean two things: the content is accurate enough to answer from, and access to it is right before the AI can surface it.
Failure Four: No Agreed Measure of Success
MD Anderson and IBM Watson, 2016 audit. In 2012, the University of Texas MD Anderson Cancer Center started building an oncology advisory tool on IBM Watson. A University of Texas System audit in 2016 found the project had cost about $62 million, including roughly $39 million paid to IBM and $23 million to PwC, and that procurement had gone around standard processes. The contract was allowed to expire in 2016, and the system was never used to treat patients.
Four years and tens of millions of dollars is what a program can look like when it has a vision but no decision points. Without an agreed measure, a baseline and a date when someone has to decide to scale, fix or stop, nothing forces the question, and every review comes back “promising, needs more time.” We don’t think a pilot without a stop rule really counts as a pilot. It’s an open-ended commitment that nobody has admitted to making.
The 2026 Evidence Points the Same Way
These four cases are well known because they happened in public. This year’s survey data shows the same pattern in programs that never make the news.
Sources: Informatica CDO Insights, January 2026; Gartner, April 2026.
Informatica’s CDO Insights survey, published in January 2026, found that 76% of data leaders say their company’s AI governance hasn’t fully kept pace with how employees use AI, and 57% say poor data reliability is keeping AI from reaching production. In Gartner’s April 2026 survey of IT infrastructure and operations leaders, 38% of those reporting AI setbacks blamed persistent skills gaps and 38% blamed poor data quality or limited data availability.
Neither one puts model capability at the top, and data readiness shows up in both. The skills gap also ties closely to ownership, since if nobody owns a program, nobody gets trained to run it. The good news is that because these failure modes are predictable, you can check for them before launch instead of discovering them afterward.
Getting Started: Run a Pre-Mortem
A pre-mortem flips the usual review around. Before launch, the team assumes the project has already failed and asks why. Since the four failure modes are well understood, we’ve found you can check most of the answers in an afternoon.
- Name the business owner, and the route for wrong answers. One person accountable for the outcome, plus a set path for catching and correcting an error. If the honest answer is “IT,” keep looking. IT owns the guardrails, not the outcome.
- Write the process down before automating it. Steps, inputs, exceptions and who handles each. Simplify what you can first.
- Review the data in scope. Check accuracy, permissions and sensitivity labels on everything the AI will be able to reach.
- Agree on one outcome measure and take a baseline. Cycle time, volume per person, rework rate or backfills avoided, rather than log-ins.
- Set the decision date. Thirty to ninety days out, with scale, fix and stop all on the table.
If any of the five has no clear answer, that’s your finding. It’s a lot cheaper to find it now than in a tribunal ruling, a news investigation or an audit.
Talk to Us
CB5 Solutions helps organizations understand the layers of AI, get the most from what they buy, and build an AI program that pays back inside their IT standards. We start every engagement with one business problem and a governance baseline. Let’s pick yours.
Next, this afternoon: Part 4, “Copilot Inherits Your Permissions: Fix Data Access Before You Deploy” — why oversharing becomes AI risk, and how to close it before rollout.