Why most AI projects fail, and it is rarely the model

More than 80% of AI projects fail. That is twice the failure rate of normal IT projects, according to RAND. And the reason is rarely the model.

Gartner found 63% of organisations do not have, or are not sure they have, the right data practices for AI. Their prediction: through 2026, 60% of AI projects without AI-ready data will be abandoned.

01 · The storyThe demo works. Production does not

I have spent more than two decades building software, and the last few years leading AI work on live SaaS products. The pattern repeats. The proof of concept is impressive on ten hand-picked questions. Then real users arrive with real data, real volume and a real workflow, and the assistant that wowed the steering committee quietly stops being used.

Gartner’s own list of why generative AI projects get dropped after the proof of concept: poor data quality, weak risk controls, rising costs, unclear business value. Not one of those is a model problem. All four are engineering and product problems.

02 · The problemWhat actually breaks

1. Retrieval: "the AI is hallucinating"

A RAG answer is only as good as the query behind it. Slow queries, missing indexes and stale data look exactly like "the AI is hallucinating". The model answered faithfully from the wrong or outdated text it was handed.

2. Latency: the old workflow was instant

Users will wait for a report. They will not wait for an assistant inside a workflow that used to be instant. They judge the AI against the old screen, not against a blank page.

3. Cost: you find out on the invoice

Token spend scales with usage. Without per-tenant logging, you find out on the invoice, after the most active customer has already used a month of budget in a week.

4. Trust: a black box gets ignored

"AI proposes, the user approves" gets adopted. A black-box answer gets ignored, and one confident wrong answer can end the project.

03 · A real caseAsk AI on a live SaaS product

At AgilityInsights we added "Ask AI" to a product customers already use every day. It answers from PDF, Word, Excel and PowerPoint documents and from live SQL. The parts that made it work were the four above, not a bigger model: documents chunked and embedded so answers draw on the right passages, generated SQL validated before it runs, every model or prompt change evaluated and traced before release.

04 · The checklistBefore your AI feature goes live

• A test set of real questions with known answers, run on every change
• Retrieval measured on its own, separately from the answers
• Indexes and query plans checked for every retrieval query
• Documents re-indexed on change, replaced versions filtered out
• A latency budget per step, tracked on the slow tail
• Cost logged per request, tenant and feature, with budgets and alerts
• Access enforced in the data layer, from the user’s login
• Sources shown, and user approval before anything changes data

The companies that win with AI will not have the best model. They will have the cleanest data, the fastest APIs and the clearest use case.

Written by

Anuj Gupta

Hands-on software engineer and architect. 23+ years writing enterprise software on .NET, Azure and Oracle Cloud, faster with AI. 21,669 hours on Upwork at a 5.0 rating.

Building something like this? Hire me LinkedInUpwork