18 min

Automatic Checks & Metrics

Format checks, matching, precision and recall, and search measures such as recall at k.

Lesson 4 of 10

0 done

In plain English

Code checks are cheap, fast and exact. Check the shape first: format, allowed values, length and required parts. Know precision and recall, and pick which matters more. Score search with recall at k. For open text, code is not enough.

Precision: Of the items picked, how many were right.
Recall: Of the true items, how many were found.
Recall at k: How often a right document appears in the top k search results.

Picture it

Checks, cheapest first

People

For high stakes and calibration

Model judge

For open qualities

Matching and metrics

Known right answers

Shape checks in code

Format, length, required parts

The cheapest, fastest and most reliable checks are written in code. They do not tire and they do not argue. This lesson covers the automatic checks that every evaluation engineer should know: format checks, exact and fuzzy matching, and the main measures for search quality. It also shows where code checks stop being enough.

Check the shape first

Before you judge meaning, check the shape. Many failures are simple. The output is not valid JSON. A field is missing. A value is outside the allowed list. The text is too long. A required citation is absent.

Schema check: does the output match the expected structure and types?
Allowed values: is each category from the approved list?
Length and format: word count, line count, required headings.
Required parts: citations, disclaimers, a final answer line.
Banned content: forbidden words, private data patterns such as card numbers.

Compare to a known answer

Exact match: the output equals the expected value. Good for labels, numbers and ids.
Normalized match: ignore case, spaces and punctuation before comparing.
Contains checks: the answer must include certain facts or phrases.
Fuzzy match: allow small differences, such as similar spelling.
Set overlap: compare lists of items, and count what was found and what was missed.

Precision and recall in plain words

Two measures appear everywhere. Precision asks: of the items the system picked, how many were right? Recall asks: of the items it should have picked, how many did it find? Choose which one matters more for the task.

precision-recall.md

Task: find all invoices that need manual review.
There are 10 invoices that truly need review.
The system flags 8 invoices. Of those, 6 truly need review.

PRECISION = right flags / all flags      = 6 / 8  = 0.75
RECALL    = right flags / true cases     = 6 / 10 = 0.60

If missing a bad invoice is costly: raise RECALL (flag more, accept more false alarms).
If false alarms waste people's time: raise PRECISION.

Measures for search

When a system retrieves documents before answering, you can score the search step alone. For each test question, you know which documents should be found.

Recall at k: in what share of questions does a right document appear in the top k results?
Precision at k: of the top k results, how many are relevant?
Mean reciprocal rank: how high up the first right document appears, averaged over questions.
Hit rate for must-find documents: did the system find the one policy page it needs?

If search fails, the answer will fail no matter how good the model is. Scoring search separately tells you where to fix things.

Where code checks stop

Code cannot judge whether a summary is faithful, a reply is polite or an explanation is clear. For open text, you need people or a model judge. Use code checks as the first layer. They catch the obvious failures cheaply. Then spend human effort where it is needed.

Keep checks honest

Test the checks themselves on a few cases where you know the answer.
Avoid rules so strict that good answers fail, such as exact match on free text.
Log which check failed, so reports show why and not only that.
Report each check separately before you combine them into one score.

→Write the code checks before you build the model judge. Many teams discover that half their failures were simple format problems.

Key takeaways

✓Check the shape first: schema, allowed values, length and required parts.
✓Use exact, normalized and fuzzy matching, and set overlap, for answers with a known right value.
✓Know precision and recall, and pick which matters more. Score search with recall at k and mean reciprocal rank.
✓Code checks are the first layer. Open text needs people or judges. Test and log your checks.

Hands-on exercise

Write five checks

Pick an AI task that returns JSON, such as extracting an invoice total and date.
Write five checks in plain words: schema, allowed values, a range check, a required field and a banned pattern.
Make ten sample outputs, some wrong. Mark which checks each one fails.
Work out precision and recall for a flagging task you invent with ten cases.

Check yourself

Q1What is the difference between precision and recall?show answer

Precision asks how many of the picked items were right. Recall asks how many of the true items were found.

Q2Why score the search step separately?show answer

If search misses the right document, the answer fails whatever the model does. Separate scores show where to fix.

Q3When do code checks stop being enough?show answer

For open text such as faithfulness, tone or clarity. Those need people or a model judge.

← Building Test Sets & Golden Data Model Judges & Human Review →