Fairness Metrics in Machine Learning: Equal Opportunity & More

Unfairness in ML occurs when models exhibit discriminatory bias, perpetuate inequality, or perform worse for disadvantaged groups. Wrongful discrimination involves disadvantaging protected classes based on characteristics like age, disability, sex, and race. Two categories exist: disparate treatment (using protected class membership as input) and disparate impact (disproportionately disadvantaged outcomes). Five fairness metrics address this: group unawareness (excluding sensitive attributes), group threshold (adjusting confidence thresholds per group), demographic parity (equal positive rates), equal opportunity (equal true positive rates among qualified individuals), and equal accuracy (equal overall accuracy). Each metric has trade-offs, and researchers have proven it's impossible to satisfy all simultaneously due to the impossibility theorem.

There are dozens of ways to define fairness (equality of opportunity, equality of outcome, minimizing false positives), but impossibility theorems show no system can satisfy all fairness definitions simultaneously. Maximizing one form of fairness results in suboptimal performance in others. In fairness scenarios, people show similar judgments for humans and machines on many dimensions, but prefer replacing machines that got it right with humans over humans that got it wrong with machines. This reveals strong machine aversion. The findings align with Title VII of the 1964 Civil Rights Act and the 1971 Supreme Court ruling that discrimination is illegal regardless of intent.

Fairness in AI requires choosing between competing metrics: equality (treating everyone identically) versus equity (different treatment based on circumstances). Key metrics include False Positive Rate, False Negative Rate, Precision, Recall, and Equalized Odds. The COMPAS recidivism case demonstrated how proxy variables can perpetuate bias despite removing explicit protected characteristics. Mathematically, it is impossible to simultaneously satisfy all fairness definitions because optimizing one metric affects another—this creates a fundamental tension requiring careful context-specific metric selection.

This extensive section presents eleven statistical criteria proposed for algorithmic fairness: calibration within groups, equal false positive/negative rates, balance for positive/negative classes, equal positive/negative predictive value, equal ratios of false positive to false negative rates, equal overall error rates, equal ratios of predicted to actual positives, and statistical parity. It then examines impossibility theorems showing that multiple criteria cannot be jointly satisfied except in marginal cases where base rates are equal or prediction is perfect. Two interpretive responses emerge: pessimistic claims that fairness is impossible except in marginal cases, suggesting inevitable unfairness; and optimistic interpretations arguing these theorems reveal which criteria are genuine fairness conditions versus statistical artifacts that can be violated by fair algorithms.

Mathematical impossibility results demonstrate that different fairness definitions cannot be satisfied simultaneously. No prediction model can achieve both equalized odds (predicting equally well across groups) and demographic parity (predicting the same average value across groups). This occurs because the data distribution creates fundamental conflicts between fairness criteria. The impossibility result means fairness enforcement requires choosing among competing principles rather than achieving perfect fairness across all dimensions, challenging simplistic notions of algorithmic fairness.

The fairness criteria based on conditional independence statements can be generalized to arbitrary random variables, including discrete group membership variables, multi-class outcome variables, and continuous variables. This means the same conceptual framework applies to multi-class classification and regression tasks, not just binary classification. The conditional independence formulations provide a unified way to express fairness requirements across different types of prediction problems.

This section extends fairness criteria to regression problems with continuous responses. Demographic parity in regression requires that the distribution of predictions be statistically independent of the sensitive attribute. Equal opportunity extends to separation, requiring predictions to be independent of the sensitive attribute conditional on the actual outcome. A third criterion called sufficiency requires that the sensitive attribute and outcome be independent conditional on the prediction, relating to group-wise calibration. The section emphasizes that different fairness criteria reflect different ethical notions of fairness and must be chosen based on the specific application context.

Multiple fairness metrics exist based on different philosophical assumptions. Statistical parity difference compares favorable outcome rates between groups. Equal opportunity difference evaluates true positive rates. Individual fairness ignores protected attributes entirely. These metrics can evaluate datasets without trained models, allowing practitioners to assess inherent bias before building models and compare how different estimators affect fairness.

Beyond base rates, other fairness definitions include: (1) Classification accuracy fairness, which considers whether classification errors are distributed equally across groups; (2) Calibration fairness, which concerns probabilistic classifiers ensuring that predicted probabilities match actual outcomes; (3) Counterfactual fairness, which requires decisions to remain the same whether an individual belongs to one demographic group or another, using causal techniques to express counterfactual scenarios.

This section covers practical estimation of fair classifiers and their desirable properties. The double plugin estimator involves: (1) estimating the fair regression prediction function, (2) estimating performance measure parameters, and (3) estimating the threshold (either explicitly or via fixed-point equation). The resulting classifier outputs 1 if the estimated prediction exceeds the estimated threshold. Two key properties are preserved: rational ordering (better candidates are not rejected while worse candidates are accepted) and monotonicity (increasing selectivity does not accept previously refused candidates or refuse previously accepted ones). These properties ensure fairness correction does not introduce perverse incentives or arbitrary rejections.

Auditing algorithmic fairness requires systematic workflow practices. During data discovery, practitioners must recognize biased sampling and sampling context—criminal datasets often reflect biased policing enforcement. When predicting low-incidence events, supplementary data from non-violent crimes may introduce racial bias. Under-weighting or over-weighting protected groups causes many machine learning algorithms to favor majority behavior patterns. Technical solutions include data massaging (label flipping, weight adjustment, corrective sampling), which research shows can reduce discrimination by factors of 8 while maintaining high accuracy. The tension between fairness and accuracy is often overstated—effective fairness algorithms can achieve significant discrimination reduction with minimal accuracy trade-offs.

Algorithmic auditing is a three-phase methodology for ensuring ethical AI development: Phase 1 involves understanding how social problems are converted into data processing problems, including identifying what data is needed and recognizing historical biases in training data; Phase 2 focuses on technical auditing to observe how algorithms impact different groups; Phase 3 addresses the integration of algorithmic decisions into organizational processes and human oversight. This approach emphasizes that ethical principles like transparency, non-discrimination, and equity must be translated into concrete technical specifications, and that effective auditing requires interdisciplinary collaboration between technical and social science perspectives to address the complex social dynamics encoded in algorithms.

IBM's AI Fairness 360 toolbox provides a comprehensive framework for auditing and improving AI fairness, containing over 30 metrics covering group fairness (selection rates, equal opportunity) and individual fairness (consistency), plus sample distortion metrics. Approximately 10 bias mitigation algorithms span pre-processing (data modification), in-processing (fairness-constrained training), and post-processing (prediction adjustment). The toolbox supports multiple datasets (COMPAS, German credit, Adult census income) and integrates with common machine learning workflows. A practical workflow involves loading data, specifying protected groups, computing baseline metrics, training models, applying mitigation algorithms, and comparing results to assess fairness-accuracy trade-offs.

The EU AI Act requires algorithms to algorithmically prove exactly what they relied on at each step, transforming compliance from paper policies to continuous machine-readable logs. Companies must provide machine-readable audit trails showing which database cells an algorithm read and whether it had legal right to do so. This breaks old corporate architecture where copying data loses context and access rights. Zero Trust approaches analyze information directly in original storage, issuing cryptographic keys for fractions of a second and recording each micro-step in immutable ledgers.

O'Neil proposes an 'FDA for algorithms' requiring testing for efficacy and safety before deployment. Currently, companies deploying algorithms onto the public can do whatever they want with no oversight. Algorithmic auditing involves 'black box techniques' - demonstrating algorithms are unfair without understanding how they work internally. Auditors measure fairness in multiple ways and show algorithms fail across the board for certain groups. The auditing process involves identifying stakeholder groups and testing for whom the algorithm fails. Problems can arise from bias data, missing data, or deployment methods, and we cannot know a priori where problems will come from.