How Algorithmic Bias Harms Facial Recognition | Joy Buolamwini

The EU's Artificial Intelligence Act establishes a pioneering risk-based regulatory framework that classifies AI systems into four levels of risk—unacceptable, high-risk, limited-risk, and minimal-risk—with corresponding transparency requirements; while banning high-risk biometric technologies like facial recognition for law enforcement except in terrorism cases, it places generative AI at a lower risk level requiring only watermarks, acknowledging the fundamental challenge of regulating rapidly evolving technology that can become obsolete within months of legislation.

The European Union is introducing an AI Act to establish coherent legal frameworks for creating AI-based products and services, their market introduction, and risk management. The Act is based on GDPR (General Data Protection Regulation) principles. The legislation prohibits systems that use subliminal manipulative techniques or misleading methods, as well as biometric categorization systems that classify individuals based on protected or sensitive characteristics. The Act also bans social scoring systems. The speaker notes that these regulations address existing technologies that were already in use before AI development accelerated, and that the EU is creating laws in real-time to keep pace with technological advancement.

The EU AI Act establishes a comprehensive regulatory framework balancing innovation with safety. High-risk AI requirements for biometrics, critical infrastructure, and law enforcement are suspended until December 2, 2027, with full implementation by August 2, 2028. The Act transfers machinery regulation from Annex 1A to 1B to eliminate duplication. Classification criteria are clarified to prevent inappropriate categorization of everyday devices. The governance structure establishes exclusive competence for the European Commission and AI Office regarding AI systems integrated into VOPS systems, while maintaining national authority coordination. Approximately 12 sectoral legislations are integrated into the AI regulatory framework. The framework aims to provide future-proof, clear, and less bureaucratic rules that can serve as a global model while enabling European companies to compete in the global technology race.

The EU AI Act represents the world's first comprehensive regulatory framework for AI across all life domains. The regulation harmonizes rules for placing AI systems on the market, putting them into operation, and using them, including prohibitions on certain AI practices, requirements for high-risk AI systems, and obligations for manufacturers and operators. The regulation applies to providers who develop AI systems, place them on the EU market, or put them into operation, as well as users who employ AI systems for professional activities. The risk-based classification system includes four tiers: unacceptable risk (generally prohibited), high risk (stringent requirements), limited risk (voluntary compliance), and minimal risk (no requirements). Prohibited practices include social scoring, exploitation of vulnerable groups, and subtle manipulation of human consciousness. High-risk AI systems include products under EU regulations and systems deployed in education, employment, financial services, and law enforcement. Transparency requirements mandate informing users when interacting with AI systems, when AI is used for emotion recognition or biometric categorization, and when deepfakes are used. Sanctions for violations are modeled after the GDPR, with fines up to 20 million euros or 4% of worldwide annual turnover.

The EU AI Act represents a comprehensive regulatory framework for AI systems, developed through years of debate and consultation. The Act employs a risk-based approach with four risk categories: (1) Unacceptable risk - systems like biometric surveillance in public spaces, which are prohibited; (2) High risk - systems affecting employment decisions, credit scoring, or emergency services, requiring strict compliance measures including transparency, impact assessments, and cybersecurity evaluations; (3) Limited risk - systems requiring specific transparency obligations; (4) Minimal risk - most AI applications with no special requirements. The Act applies to any AI system deployed in the EU market, regardless of where it was developed, similar to how GDPR applies to data processing.

There is no universal bias mitigation technique, as different types of biases require different strategies. The goal is to reduce bias rather than eliminate it completely. Three technical approaches exist: (1) Pre-processing - improving data before training through fair representation, optimized pre-processing, removing disparity, improving sampling methods, reweighting, and rebalancing data via oversampling or undersampling; (2) In-processing - modifying learning algorithms during training through fairness-aware loss functions, adversarial debiasing, hyperparameter optimization, and calibration; (3) Post-processing - adjusting outputs after training through threshold adjustment, calibration, group-specific correction, and fairness-aware relabeling.

Synthetic data generation addresses privacy concerns and dataset limitations in fairness research. Approaches include heuristic generation, matrix factorization with autoencoders, and data obfuscation. For fairness-aware experiments, synthetic data is generated by creating user and item matrices, computing latent factors, and injecting bias or matching real-world distributions. This enables testing dynamic fairness and longtail issues that real-world datasets cannot adequately support, particularly for applications requiring timestamped user arrival patterns.

Three main approaches exist for mitigating unfair biases: (1) Pre-processing - modify observed data before training (e.g., learn representations independent of sensitive attributes); (2) Post-processing - modify statistical outputs after training; (3) In-processing - retrain the model subject to fairness constraints via constrained optimization. Each approach has trade-offs: pre-processing is agnostic to fairness notion but may lose information; post-processing is quick but limited to specific fairness notions; in-processing is flexible but computationally intensive.

This section covers three major approaches for implementing fairness in machine learning systems. Pre-processing transforms feature representations to remove sensitive attribute dependence before training, using linear models, kernel methods, or adversarial networks. In-processing integrates fairness constraints into model training via regularized empirical risk minimization, balancing accuracy, complexity, and fairness through optimization techniques. Post-processing modifies existing deployed models by adjusting decision thresholds or applying transformations based on optimal transport theory or conformal prediction. Each approach offers different trade-offs between flexibility, computational cost, and fairness guarantees, with the choice depending on whether fairness needs to be incorporated at design time, during training, or after deployment.

Fairness-aware machine learning frames the problem as optimizing for accuracy and fairness simultaneously, often as a multi-objective optimization problem. However, this approach inherits all the problems of traditional ML optimization, including shortcuts and side effects. Known issues include 'leveling down' (reducing performance for all groups to achieve fairness) and strange selection behaviors where models find shortcuts to optimization goals that have little practical sense. The challenge is developing methodologies that can spot these side effects during model design and testing, rather than discovering them years later.

AI systems often perform differently across intersecting demographic categories. In the Gender Shades study, all three companies showed lower accuracy for darker-skinned females compared to other groups. IBM had the largest accuracy gap at 34 percentage points between lighter males and darker females. This intersectional analysis reveals that simply testing by individual factors like gender or skin type separately is insufficient to understand systemic bias.

Intersectionality examines how different characteristics (gender, race, age, economic level, sexual orientation) combine to create unique experiences of discrimination or privilege. When applying this to AI, we examine how systems can reproduce or amplify inequalities, especially when training data does not adequately represent all groups. Data is not always neutral - if trained with data reflecting social inequalities, AI can learn and repeat these biases. Facial recognition systems work well for light-skinned faces but poorly for dark-skinned faces because most training data consists of light-skinned faces. AI design should consider intersectional factors rather than treating characteristics in isolation.

Applying Kimberlé Crenshaw's intersectional framework to AI research reveals biases invisible in single-axis analyses. While aggregate accuracy metrics may appear acceptable (e.g., 90% overall), breaking down results by gender and skin type exposes stark disparities. All major facial analysis systems perform better on lighter-skinned faces than darker-skinned faces, and better on male faces than female faces. However, the intersection of both factors reveals that darker-skinned women consistently experience the worst performance across all tested companies. Follow-up research showed some targeted companies improved their accuracy gaps, while non-targeted companies maintained similar problematic performance levels, demonstrating that external accountability drives meaningful change in algorithmic fairness.

Intersectionality is a concept from social theory (specifically feminist theory) that examines how different social identities (such as race, gender, class) intersect to create unique experiences of discrimination or privilege. In AI research, this term was used to describe studies that analyze bias against multiple intersecting identities simultaneously, rather than examining single-axis biases.

Kimberlé Crenshaw's intersectional theory shows that experiences of someone who is both Black and female cannot be understood by simply combining experiences of being Black and being female separately. Unique experiences emerge from the intersection of multiple marginalized characteristics. AI evaluation must consider these intersectional identities explicitly rather than analyzing single dimensions independently.