19 min

Fairness, Transparency & Explainability

Where bias comes from, a simple fairness check, honest transparency and the documents that create accountability.

Lesson 6 of 10

0 done

In plain English

Bias can come from data, labels, design, use and feedback loops. Fairness has several definitions, so make the choice visible, test against it and record it. Be open with users, and keep system cards, impact assessments and change logs up to date. State what you did not test.

Bias: Unfair differences in how a system treats groups.
System card: A short record of a system's purpose, limits and testing.
Explainability: How well people can understand why a system gave a result.

Picture it

A simple fairness check

Pick the groups

Who might be treated differently

Compare outcomes

Error and approval rates

Find the cause

Data, labels, design or use

Decide and record

Accept, reduce or stop

Repeat on a schedule

Data and use drift

AI can treat groups of people differently without anyone meaning it to. It can also be a black box that nobody can explain. Governance asks three things. Is it fair? Can we explain it? Can we show how it was built and tested? This lesson covers fairness checks, transparency to users and the documents that make AI accountable.

Where bias comes from

Data: past records may reflect past unfairness, or may under-represent some groups.
Labels: the people who label data bring their own views.
Design: the goal or the measure may favor some groups.
Use: a tool built for one setting is used in another, where it works worse.
Feedback loops: the system's choices change the data it later learns from.

Fairness is a choice, not a switch

There are several ways to define fair, and they can conflict. Equal error rates across groups is one. Equal chance of a good outcome for similar people is another. A governance lead cannot pick one for everyone. The job is to make the choice visible, decide it with the business and legal teams, test against it and write it down.

A simple fairness check

fairness-check.md

1. DECIDE WHO MIGHT BE TREATED DIFFERENTLY
   Groups that matter for this use and the law: e.g. age, sex, disability, language, region.
   (Some data may be restricted by law; ask privacy and legal before collecting it.)

2. COMPARE OUTCOMES
   For each group, measure what matters: approval rate, error rate, false alarms, misses,
   quality of answers, response time.

3. LOOK FOR GAPS
   Are some groups clearly worse off? Is the gap larger than chance would explain?
   Check sample sizes before you read much into small groups.

4. FIND THE CAUSE
   Data? Labels? Design? Use? Fix the cause, not only the number.

5. DECIDE AND RECORD
   Accept, reduce, or stop. Write down the choice, who made it and why.

6. REPEAT
   After changes and on a schedule, because data and use drift.

Transparency to users

Tell people when they deal with an AI, where rules require it or where it affects trust.
Say what the AI can and cannot do, in plain words.
Label AI-generated content where it matters.
Give people a way to reach a person and to challenge a decision that affects them.
Explain the main reasons behind a decision about someone when the law or fairness calls for it.

Explainability

Some systems are easy to explain, such as a simple rule. Language models are harder. You can often explain what data the system saw, which sources it cited and which rules it applied around the model. Do not claim that a model explanation of its own reasoning is a true account. It may be a plausible story. Rely on records, sources and tests.

Documents that create accountability

system-documents.md

SYSTEM CARD (for each AI system)
  Purpose and limits; intended and not-intended uses; data used; how it was tested;
  known weaknesses; oversight design; owner; version; date

MODEL CARD (from the vendor, or written for your own model)
  Model type, training data summary, evaluation results, known limits, safe-use notes

IMPACT ASSESSMENT (for high-risk uses)
  Who is affected; benefits and harms; fairness and privacy checks; controls; decision and sign-off

CHANGE LOG
  What changed, when, why, who approved, what was retested

⚠Do not write documents for the audit only. A system card that nobody updates or reads is worse than none, because it gives false comfort.

Be honest about limits

Say what you did not test. Say which groups you could not measure. Say how the system may fail. Honest limits protect users and protect you. They also make good decisions possible.

Key takeaways

✓Bias can come from data, labels, design, use and feedback loops.
✓Fairness has several definitions that can conflict. Make the choice visible, test against it and record it.
✓Be transparent: tell people about AI use, state limits, offer a route to a person and explain important decisions.
✓Keep system cards, model cards, impact assessments and change logs up to date, and state what you did not test.

Hands-on exercise

Run a small fairness check

Imagine an AI that pre-screens support requests, sending some to a fast lane.
Pick three groups to compare, such as language, region and customer type, and say why.
Invent a small table of outcomes for each group. Look for gaps and write one possible cause.
Write the decision you would take, and the entry for the system card.

Check yourself

Q1Why is fairness a choice and not a switch?show answer

There are several definitions of fair, and they can conflict. The organization must choose, test and record its choice.

Q2Why not trust a model's explanation of its own reasoning?show answer

It may be a plausible story rather than a true account. Rely on records, sources and tests.

Q3What is a system card?show answer

A short record of an AI system's purpose, limits, data, testing, weaknesses, oversight and owner.

← Data Privacy & Intellectual Property Risks Third-Party & Vendor Risk →