Why Fixing One AI Bias Can Create Another
Making an AI system fair is not always like correcting a typo or deleting one bad rule.
Equal outcomes, equal error rates, and equally reliable risk scores can all sound fair. Under some conditions, however, a system cannot satisfy all of them at once.
The first article followed bias through the AI pipeline. This article examines a harder problem: people can agree that a system should be fair while disagreeing about what fairness should mean mathematically.
Imagine an AI system that helps rank applications for additional review.
After testing it, a team discovers that two groups receive different approval rates.
The team changes the system until the approval rates become equal.
Has the bias been fixed?
Perhaps one gap has been reduced. But another measurement may now look worse.
Qualified applicants in one group may be missed more often. Incorrect approvals may rise in another group. A score of 70% may no longer represent the same level of risk for everyone.
The problem is that fairness has several definitions.
Several different results can sound fair
Consider these statements:
- Everyone should be judged using the same rule.
- Each group should receive positive decisions at the same rate.
- Qualified people should have the same chance of being selected.
- People who do not meet the criteria should face the same false-alarm rate.
- The same risk score should mean the same thing for every group.
Each statement expresses a recognizable idea of fairness.
They do not always describe the same system.
In some statistical settings, it is impossible to satisfy several of these conditions simultaneously unless special conditions hold—for example, unless relevant outcome rates are already the same or the model predicts perfectly.
Fairness definitions are not universally incompatible in every dataset. The conflict appears under particular conditions, especially when groups have different recorded outcome rates and the model makes mistakes.
Begin with false positives and false negatives
A classification model often makes two broad kinds of error.
A false positive occurs when the system predicts a positive result for a case that is actually negative.
A false negative occurs when the system predicts a negative result for a case that is actually positive.
For a fraud-detection system:
- a false positive blocks a legitimate transaction
- a false negative allows a fraudulent transaction through
Reducing one kind of error often increases the other.
If the system blocks more transactions, it may catch additional fraud but also interrupt more legitimate purchases. If it becomes more permissive, it may inconvenience fewer customers while missing more fraud.
The threshold changes the balance.
Fairness definition 1: Equal outcomes
One approach asks whether groups receive positive outcomes at similar rates.
In an application system, this might mean that each group advances at the same percentage.
This is often called a form of demographic parity.
It measures the distribution of decisions. It does not ask whether the people receiving those decisions had the same recorded outcomes or qualifications.
Equalizing the selection rate can correct a visible disparity.
However, it may not equalize false positives, false negatives, or the accuracy of the final decision.
Fairness definition 2: Equal opportunity
Another approach focuses on cases that truly belong in the positive category.
It asks whether qualified people from different groups have a similar chance of receiving a positive prediction.
For a medical screening system, the question might be:
Among patients who actually have the condition, does the model detect it at similar rates across groups?
This focuses attention on false negatives.
If a condition is missed more often for one population, equal overall accuracy may not provide equal opportunity to receive treatment.
Fairness definition 3: Equal false-alarm rates
In other applications, false positives may be especially harmful.
A legitimate bank transaction may be blocked. An acceptable social-media post may be removed. A person may be incorrectly sent for additional security screening.
A fairness goal may therefore require similar false-positive rates across groups.
But adjusting a threshold to equalize false positives can change the false-negative rate.
The system may stop producing one uneven error while producing another.
Fairness definition 4: Calibrated scores
Some systems return a risk score rather than a simple yes-or-no decision.
Suppose a score of 70 means that around 70 out of 100 similar cases experience the predicted outcome.
If the score has that meaning across groups, it is described as calibrated across those groups.
Calibration can be useful because equal scores then carry a similar statistical interpretation.
However, when recorded outcome rates differ and predictions are imperfect, maintaining calibration can conflict with making both false-positive and false-negative rates equal.
A system can satisfy one reasonable fairness rule and still fail another reasonable fairness rule. That does not automatically mean the engineers made a simple mistake.
A plain-English example
Imagine two groups of 1,000 applications.
In the available historical records, the measured positive outcome appears at different rates in the two groups.
Those recorded differences may reflect real conditions, earlier policies, unequal access, measurement problems, or a mixture of causes.
If the model uses one threshold for everyone, it may produce different false-positive and false-negative rates.
If the thresholds are changed to equalize those error rates, people with similar model scores may receive different decisions.
If selection rates are forced to match, accuracy or calibration may change.
There is no guarantee that one setting satisfies every goal.
Why the seesaw analogy is useful—and limited
Fairness trade-offs are sometimes compared with a seesaw.
Push one side down and another side rises.
The analogy captures the fact that changing a threshold or objective can improve one measurement while worsening another.
But it should not be taken as a universal law that every fairness improvement harms someone else.
Better training data, improved measurements, more appropriate features, or a better model can sometimes improve several groups at once.
The strongest conflicts appear when teams try to force several specific fairness conditions onto an imperfect prediction system under particular statistical conditions.
Removing a protected field does not solve the trade-off
A team may remove race, gender, age, or another sensitive characteristic from the model’s input.
That may be appropriate in some settings, but it does not automatically make the model fair.
Other features may correlate with the removed characteristic. Location, language patterns, education history, purchasing behavior, or device type may carry related information.
Removing the field can also make evaluation harder if teams no longer have the information needed to compare error rates.
The ethical and legal handling of protected information depends on the application and jurisdiction. But from a technical perspective, simply deleting one column does not prove that uneven behavior has disappeared.
Using the same rule can still produce different results
Applying one threshold to everyone can feel fair because the rule is identical.
But identical rules do not guarantee identical error rates.
Suppose a speech-recognition model is less accurate for an accent that appeared less often in training.
Using the same confidence threshold for every speaker does not correct the weaker recognition quality.
The system treats the score identically while the score itself is less reliable for some users.
The application determines which error matters most
A movie recommendation, cancer-screening tool, fraud detector, hiring system, and criminal-justice model do not create the same consequences.
A poor film recommendation may waste two hours.
A missed diagnosis can delay treatment.
A false fraud alert can block access to money.
A mistaken high-risk classification can affect a person’s liberty.
Fairness decisions should therefore consider:
- the severity of each error
- whether the error can be reversed
- who bears the cost
- whether human review is available
- whether affected people can appeal
Mathematics measures trade-offs but does not choose values
Engineers can calculate false-positive rates, false-negative rates, selection rates, and calibration.
Those numbers show how a system behaves.
They do not decide which harm society should prioritize.
Choosing the fairness goal is partly a technical decision, but it is also a legal, ethical, institutional, and political decision.
The people affected by the system should not be excluded from that choice.
A stronger correction process
Instead of issuing the vague instruction “remove the bias,” a team can ask:
- What specific harm has been observed?
- Which group or condition is affected?
- Is the problem a false-positive gap, false-negative gap, outcome gap, or calibration problem?
- Does the difference come from data quality, labels, the model, or deployment?
- Which fairness goal fits the purpose of the system?
- What other measurements change after the correction?
This turns fairness into a testable engineering and governance problem rather than a slogan.
Several definitions of fairness can conflict when groups have different recorded outcome rates and the model makes mistakes. Improving one measure can therefore worsen another, although better data and modeling may sometimes improve several measures together.
Fairness is not one mathematical dial
An AI system does not contain a single control marked “fair” or “unfair.”
It contains data, scores, thresholds, objectives, and decisions about which errors matter.
Correcting bias requires stating the fairness goal clearly and checking what the correction changes elsewhere.
The difficult question is not only how to adjust the model.
It is which form of fairness the system should protect, and why.
Next in the series: The next article explains how a model can produce biased results without receiving an explicit biased rule—and how ordinary variables can act as statistical shadows of hidden characteristics.
Comments
Post a Comment