Skip to content

AI ROI: Why Returns Stall and How to Measure Them

Brian Lambert, PhD
· 19 min read
AI ROI: Why Returns Stall and How to Measure Them

AI ROI is the financial return an AI investment produces relative to the full cost of deploying and running it. The hard part is not the formula. It is that most organizations cannot show what changed in the work, because they never defined what the work needed before they bought the tool. Returns stall when capability, not technology, is the bottleneck. They become measurable when you diagnose first, baseline early, and measure outcomes rather than logins.

TL;DR

  • AI spending is rising faster than realized value. Gartner reports AI use cases grew 400% from 2024 to 2025 while the odds of success stayed at about one in five.
  • Most organizations wait two to four years for satisfactory returns on a typical use case, well beyond the seven to 12 months expected from other technology.
  • Productivity gains are real in some roles, but they depend on how the work is redesigned around the tool.
  • Without a baseline taken before go-live, you cannot separate the effect of AI from noise.
  • Before you build training or roll out another tool, decide what capability the work requires and whether learning is even the right enablement play.

Table of Contents

Cognistry
Make AI Returns Measurable
Cognistry diagnoses the capability work requires, grounds enablement in organizational evidence, and measures outcomes back to the business.

Why AI ROI is so hard to find

The gap between activity and value is now well documented. Gartner’s analysis, More AI Investment Is Not Creating Enough AI ROI, describes investment rising faster than value. Deloitte’s research on the paradox of rising investment and elusive returns found that only 6% of organizations report payback in under a year.

Use is not the problem. The Wharton 2025 AI Adoption Report found 82% of enterprise leaders use generative AI at least weekly. Yet Gartner’s figures show only 13% of enterprises reporting significant or transformational business value.

That is the pattern. High usage. Low proof.

We call the cause Data Drag: the performance-draining gap between what people learn or are given and what they actually do differently on Monday morning. AI tools get switched on. The work around them stays the same. The tool digitizes existing inefficiency, and the P&L does not move.

Usage is not value

Most AI scorecards lead with adoption: active users, prompts sent, licenses assigned. Those numbers matter, but as a floor. HR Bench’s guidance on AI workforce transformation puts it plainly: adoption tells you where a rollout stalled, not whether it paid off.

The same source argues for tracking work redesign: how many roles were formally updated, or created, because AI changed how the work gets done. If no role, process or decision changed, you have bought a tool, not a result.

A useful test for any AI dashboard:

  1. Does the metric describe use of the tool, or a change in the work?
  2. Could it rise while the business outcome stays flat?
  3. Does a business owner, not the AI team, own the outcome it points to?

If the answer to the second question is yes, the metric is a usage metric. Keep it, but do not call it ROI.

What AI productivity gains actually look like

There is evidence that AI can lift output. In three field experiments with software developers at Microsoft, Accenture and a Fortune 100 company, developers using an AI coding assistant completed 26.08% more tasks in a pooled analysis of nearly 5,000 people. The study also reported that less experienced workers adopted the tool more and gained more.

Read that carefully before quoting it in a board deck:

  • It covers software developers, not every role.
  • The standard error on the main estimate was 10.3%, so the true gain could sit well above or below the headline.
  • Completed tasks are an output measure. They say nothing by themselves about quality, rework or customer outcomes.

An early meta-analysis of generative AI and productivity also notes substantial differences across studies. Your own results will depend on the task, the people, and how the work is set up around the tool.

That last part is where most organizations lose the gain. A tool can make a task faster. Whether the faster task changes a business result depends on decisions: what the person does with the AI output, when they override it, and who is accountable when it is wrong. That is operational judgment, and it is a capability the work has to be designed to develop.

Set a baseline before go-live

The most common reason ROI stays elusive is also the cheapest to fix. Teams launch first and measure later. Guidance for CHROs on AI ROI recommends settling the measurement framework before launching AI programs, and tracking efficiency, quality and other categories, such as time saved per task, cycle time and administrative burden.

A practical baseline looks like this:

What to capture Example measure Why it matters
Cycle time Hours from request to resolution Shows speed change attributable to the tool
Error or rework rate Share of outputs returned for correction Shows whether faster also means worse
Handling effort Hours per task, by team Converts into capacity you can price
Decision quality Escalations, overrides, quality findings Shows whether people use AI output well
Cost to run Licenses, integration, support, change effort The denominator of ROI

Collect these for a defined period before the tool goes live. One source argues for a 30-day baseline, and the principle holds whatever the length: without documented current performance, you cannot isolate the program’s effect later, and projects get declared successful on belief rather than evidence.

Take a recruiting example. A team that replaces phone screens with AI can only claim recruiter hours saved if it recorded hours per screen before the change. The same team should record quality of hire alongside speed. The reverse risk is just as real: a ranking model can look accurate while still raising fairness concerns in AI resume ranking. One number is never the whole return.

Recruiting baseline and outcome measures compared

Count the whole cost, and the whole return

Two mistakes distort the ratio.

Counting only license cost. The real denominator includes integration, data preparation, governance, support, and the time people spend learning to work differently. Leave out change effort and ROI looks better than it is, until the rollout stalls.

Judging each project alone. PwC’s view in its AI ROI analysis favors a portfolio approach: balance near-term efficiency gains with longer-term strategic value such as agility, retention and building internal expertise. Some of these returns are softer. They still count, as long as you name them in advance and say how you will recognize them.

A simple way to hold both:

  • Hard returns: time, cost, error and cycle-time changes you can tie to a baseline.
  • Strategic returns: capabilities the organization gains, written as testable statements, such as “frontline supervisors resolve exceptions without escalating.”
  • Costs: every line, including the people side.

Is it a tool problem or a capability problem?

When AI results disappoint, the reflex is to buy training. Before you build that course, ask a different question: what does the work actually require, and what is stopping people from doing it?

Sometimes the answer is knowledge. Often it is not. The cause may sit in the environment: unclear decision rights, a process that still routes around the tool, incentives that reward the old way, missing governance. No course fixes those. Training people harder inside a broken workflow produces completion certificates and flat performance.

We look at this in three layers:

  1. Environment. Does the process, the data and the accountability structure let the work change?
  2. Capability. Do people have the judgment to use AI output in real situations, including when to trust it and when to stop?
  3. Enablement. Only after the first two, decide whether learning is the right enablement play, and what form it should take.

Our pieces on why AI adoption is a capability problem, not a technology problem and on the deployment wall that blocks scaling go deeper. If you have already bought AI and performance has not moved, diagnosing first in six to eight weeks is a clear starting point.

Perspective: ROI is a design decision, not a report

We see ROI conversations go wrong in a consistent way. The organization buys the tool, builds training on it, and then asks finance to prove it worked. By then the questions that would have made ROI provable are gone: What was the work supposed to do differently? What evidence showed that was the right response? What did performance look like before?

Our position is that ROI is settled at the start. Three commitments make the difference:

  • Diagnose before you build. Establish what capability the work requires and whether learning is the right response at all. Sometimes the best enablement play is a process or governance change.
  • Ground design in your own evidence. That means your strategy, frontline friction, quality findings and the expertise of your own subject-matter experts, not generic content.
  • Measure back to the business. Course completion is an activity. A change in cycle time, rework or decision quality is a result.

Human judgment is where the value lives. As AI takes on more of the routine work, what is left for people is deciding, checking and exercising human-in-the-loop leadership. Organizations that treat that as a capability to design for will see returns sooner than those that treat it as a side effect of rollout. We would rather help you find that the answer is not training than sell you a course that cannot work.

— Brian

A measurement plan you can start this quarter

You do not need a new platform to begin. You need discipline.

  1. Name the business outcome. One sentence, owned by a business leader: “Reduce rework on claims handling,” not “Increase AI usage.”
  2. Baseline it. Capture current cycle time, error rate and effort before go-live. Document how you measured.
  3. Run a cause check. For each gap between current and target performance, ask whether it comes from the environment or from capability. Use frontline input and quality findings, not assumptions.
  4. Redesign the work. Update the process, roles and decision rights. Track how many roles actually changed.
  5. Choose the enablement play. Training, practice, job support, or no learning at all. Match it to the cause.
  6. Measure on a fixed schedule. Compare against baseline at set points, and report hard and strategic returns separately.
  7. Set a governance rule. Decide who reviews AI output, when, and what triggers a stop. A practical governance approach for U.S. organizations helps here.

If you cannot complete step 1 or 2, pause the rollout. Spending more will not help.

How Cognistry fits

Cognistry is a capability engineering platform. We start with diagnosis: what the work requires, whether learning is the right response, and what your own evidence says. From there we structure the response, whether that is courses, practice simulations or decision environments built for operational judgment, and connect outcomes back to the business.

It is not a slide authoring tool and it is not an AI course generator. If your AI program is stuck between usage and results, the Cognistry Platform is where that diagnosis, design and measurement come together. For leaders at the deployment stage, Edge focuses on activating adoption and change. You can also see how readiness checks before you build work in practice.

FAQ

What is a good ROI for AI?

There is no universal benchmark, and the research shows wide variation. Deloitte found most organizations take two to four years to reach satisfactory returns on a typical use case. A better question is whether your result beats your own baseline by an amount that covers the full cost, including change effort.

Is AI ROI real?

It is real in specific, well-defined cases. Field experiments with software developers found a 26.08% rise in completed tasks, though with a wide margin of error. At the enterprise level, however, Gartner reports that few organizations see significant business value, which usually points to weak measurement and unchanged work rather than a failed technology.

How do you measure AI productivity?

Start with a baseline, then track a change in the work: cycle time, rework, effort per task and decision quality. Usage numbers like logins show adoption, not productivity. Pair any speed gain with a quality measure so faster output does not hide more errors.

Why do most AI projects miss their ROI targets?

Common causes are no baseline before launch, success defined as usage, and workflows left unchanged around the tool. Evaluating each project in isolation also hides strategic returns. Diagnosing whether the gap is capability or environment before building anything addresses most of these.

Sources

Continue Reading