18 min

Measuring Value & Reporting to Leaders

Pick the right measures, keep a baseline, run a scorecard and review on a rhythm.

Lesson 9 of 10

0 done

In plain English

Measure business results, not only technical scores. Track use, quality and value together, and keep a baseline from before the change. Use a one-page scorecard with cost per completed task, review it on a fixed rhythm, and report to leaders with the decision you need.

Baseline: The number from before the change.
Scorecard: A one-page view of use, quality, value and cost.
Vanity number: A count that shows activity but not value.

Picture it

Measure all three, not one

Review the scorecard on a fixed rhythm and write the decision down.

What to track

Use: weekly active users
Quality: reviewer pass rate
Value: the business result

What to avoid

Counting prompts sent
No baseline from before
Counting a saving twice

A business case promises value. Measuring proves it. Without measuring, projects drift, budgets grow and nobody can say whether AI paid off. This lesson shows how to choose measures, track benefits after launch and report in a way leaders trust.

Measure what the business cares about

Start from the success number you chose in the business case. It should be a business result, not a technical one. Minutes saved per task, fewer errors, faster replies and more leads followed up are good examples. Model scores and token counts matter to engineers. Leaders care about outcomes.

Three kinds of measure

Use: how many people use the tool each week, and for how many tasks.
Quality: how good are the results, judged by reviewers on a sample.
Value: the business result, such as hours saved, cost avoided or revenue gained.

You need all three. High use with poor quality is a risk. Good quality with no use brings no value. Value without a baseline is only a claim.

Always keep a baseline

Measure the old way before the change. Without it, you cannot show improvement. If you forgot, estimate from past records, and say clearly that it is an estimate.

A simple scorecard

ai-scorecard.md

PROJECT: {{name}}        OWNER: {{name}}        REVIEW DATE: {{date}}

USE
  Weekly active users: {{n}} of {{target}}        Tasks done with the tool: {{n}}

QUALITY
  Reviewer pass rate on sample: {{x of y}}        Must-never-happen incidents: {{n}}

VALUE
  Baseline: {{number}}   Now: {{number}}   Change: {{number}}
  Benefit this month (money or time, with the method shown): {{...}}

COST
  Running cost this month: {{...}}      Review and support time: {{...}}
  Cost per completed task: {{...}}

RISKS AND ACTIONS
  {{what is off track, who acts, by when}}

Count benefit honestly

Show the method: tasks times minutes saved times hourly cost. Let others check it.
Say what happened to the saved time. If nothing changed, the saving is on paper only.
Count the full cost, including review time and support.
Do not count the same benefit twice across projects.

Review on a rhythm

Set a regular review, monthly at first and then quarterly. Look at the scorecard. Decide: keep, fix, scale or stop. Write the decision down. A project that nobody reviews will drift.

Look across the whole portfolio

A strategist also looks at all AI projects together. Which ones deliver? Which cost more than they return? Where do the same problems repeat? A simple table of projects, owner, tier, cost, value and status is enough. It helps leaders move money from weak projects to strong ones.

Report to leaders

Lead with the result and the decision you need.
Show a few numbers against the plan. Avoid long charts.
Be open about what is not working and what you will do.
Keep the format the same each time, so changes stand out.

⚠Beware of vanity numbers such as number of prompts sent or chats opened. They show activity, not value.

Key takeaways

✓Measure business results, not only technical scores. Track use, quality and value together.
✓Keep a baseline from before the change. Without it you cannot prove improvement.
✓Use a one-page scorecard with cost per completed task, and review it on a fixed rhythm.
✓Count benefits honestly, show your method, and report with the decision you need.

Hands-on exercise

Build a scorecard

Pick one AI use case. Write its baseline for one value measure.
Fill in the scorecard with imagined numbers for month three.
Write the decision you would take from it: keep, fix, scale or stop, with a reason.
Write a five-line update for a leader that opens with the result and the decision needed.

Check yourself

Q1Why do you need use, quality and value measures together?show answer

High use with poor quality is risky, good quality with no use gives no value, and value needs both. Each alone is misleading.

Q2What is a baseline?show answer

The number from before the change. It lets you show how much the project improved things.

Q3Why is number of prompts sent a weak measure?show answer

It shows activity, not value. A business result such as hours saved or errors avoided matters more.

← Data Readiness & Company Knowledge Strategy Interviews & Case Practice →