Testing AI With the Customer

Customers do not trust AI because you say it works. They trust it because they saw it work on their own cases. Testing with the customer turns a promise into evidence. This lesson shows how to build tests together, report results in plain words and keep quality steady after launch.

Why test with the customer

You do not know their work as well as they do. Only they can say whether an answer is good. If they help build the test, they also believe the result. They have already agreed what good looks like before you show a demo.

Build the test set together

• Ask the customer's experts for 30 to 100 real cases. Include easy, typical and hard ones.
• Hide private details where the rules require it.
• Ask the experts to write the right answer, or to mark answers as acceptable or not.
• Keep a separate set that you never use while tuning. Use it only for the final check.
• Add every real failure to the set as a new case.

Agree the pass mark in writing

Before you build, agree what passing means. The agreement can be one page. It prevents arguments later.

ACCEPTANCE AGREEMENT

TEST SET: {{number}} real cases, chosen with {{customer expert}}
WHO JUDGES: {{names}}; two judges per hard case
WHAT IS A PASS: {{e.g. the answer is correct and cites a source}}
PASS MARK: {{e.g. at least 9 of 10 cases pass}}
MUST NEVER HAPPEN: {{e.g. reveals another customer's data; invents a refund rule}}
  -> any single failure here means the release stops
UNSURE CASES: sent to a person; counted as a pass if the AI correctly says it is unsure
RETEST: after every change to the prompt, model or data
SIGN-OFF: {{name, date}}

Say results in plain words

Customers do not need formulas. Use simple sentences. Say, on 50 real cases, 46 were right, 3 needed a small fix and 1 was wrong. Then show the one that was wrong. Honesty builds trust.

Two terms help. Precision asks, when the AI says yes, how often is it right? Recall asks, of all the real yes cases, how many did it find? Pick which one matters more for the job. A fraud alert may prefer recall. A message that reaches customers may prefer precision.

Study the errors

A pass rate says how often. An error review says why. Read every failure. Group them into types, such as missing information, wrong rule, bad formatting or hostile input. Fix the biggest group first.

CASE | WHAT HAPPENED                         | TYPE               | FIX
014  | Gave last year's price                | Old data           | Refresh the source, add a date check
027  | Refund approved above the limit       | Missing rule       | Add limit check in code, not in the prompt
033  | Answer too long for the screen        | Format             | Add length limit to the prompt
041  | Invented a policy name                | Made-up fact       | Require a citation; say unsure otherwise

When the experts disagree

Sometimes two experts judge the same case differently. Do not hide this. It shows the rule is unclear. Bring them together, agree the rule and write it down. The AI cannot be more consistent than the people who define good.

Keep quality steady after launch

• Re-run the test set after every change to the prompt, the model or the data.
• Review a small sample of real results each week with the customer.
• Watch for drift: new products, new rules or new wording that the test set never saw.
• Keep a change log, so you can say what changed when quality moved.