Coded Bias: Facial Recognition Racism and Algorithmic Justice

The EU AI Act is the world's first comprehensive regulatory framework for artificial intelligence. It classifies AI systems by risk level: unacceptable risk systems (real-time surveillance, social scoring, behavioral manipulation) are prohibited; high-risk systems (education, employment, justice, infrastructure) require audits, human supervision, and security testing; general-purpose models must publish training data information and ensure cybersecurity. The legislation was approved in 2024 with phased implementation starting in 2026. The EU is considering suspending parts of the law due to pressure from major tech companies and US government concerns about commercial barriers.

Robust regulatory frameworks are essential for keeping technology within democratic and human rights boundaries. The European Union has adopted the Artificial Intelligence Act, though it contains exemptions that limit its effectiveness—for example, facial recognition cameras used by police forces remain largely unregulated. Good regulatory development requires action at local, national, and supranational levels. Cities like New York and San Francisco have banned facial recognition technology for law enforcement, demonstrating that stronger regulations are possible.

The EU's Artificial Intelligence Act establishes a pioneering risk-based regulatory framework that classifies AI systems into four levels of risk—unacceptable, high-risk, limited-risk, and minimal-risk—with corresponding transparency requirements; while banning high-risk biometric technologies like facial recognition for law enforcement except in terrorism cases, it places generative AI at a lower risk level requiring only watermarks, acknowledging the fundamental challenge of regulating rapidly evolving technology that can become obsolete within months of legislation.

The EU's AI Act represents the West's first comprehensive AI regulation framework, establishing 'guard rails' around high-risk AI applications such as facial recognition, predictive policing, and real-time biometric surveillance, while also addressing emerging challenges from generative AI tools like ChatGPT through requirements for preventing illegal content creation and transparency in copyrighted data usage; the legislation balances industry calls for rules against concerns about stifling innovation, with final approval requiring negotiation between the European Parliament, Commission, and member states.

The European Council has finalized AI safety rules under the AI Act, which is stronger and more comprehensive than the US voluntary compliance approach. The rules restrict EU governments from using real-time biometric monitoring like facial recognition except for locating suspects of extremely serious crimes or preventing violent terrorist attacks. Law enforcement is also banned from using AI for social scoring, predictive policing, and untargeted scraping of facial images.

There is no universal bias mitigation technique, as different types of biases require different strategies. The goal is to reduce bias rather than eliminate it completely. Three technical approaches exist: (1) Pre-processing - improving data before training through fair representation, optimized pre-processing, removing disparity, improving sampling methods, reweighting, and rebalancing data via oversampling or undersampling; (2) In-processing - modifying learning algorithms during training through fairness-aware loss functions, adversarial debiasing, hyperparameter optimization, and calibration; (3) Post-processing - adjusting outputs after training through threshold adjustment, calibration, group-specific correction, and fairness-aware relabeling.

Effective bias mitigation requires multiple technical strategies: reweighting training data based on privileged features to balance representation; gathering more diverse training data, especially for visual recognition systems where certain groups may be underrepresented; implementing adversarial classifiers that prevent the model from inferring protected attributes; and applying production-level adjustments to boost predictions for underrepresented groups. A critical insight is that mitigation actions can reduce bias without sacrificing accuracy, and in some cases, accuracy improves after bias correction. IBM's open-source tools provide integrated solutions for monitoring model accuracy, performance, and fairness across different AI technologies.

AI fairness involves addressing bias in machine learning models through three categories of mitigation algorithms: pre-processing (modifying training data), in-processing (modifying the training algorithm), and post-processing (modifying classifier outputs), with fairness metrics like statistical parity difference, disparate impact, equal opportunity difference, and average odds difference helping practitioners evaluate and reduce unwanted algorithmic bias in their models.

This section presents three categories of fairness mitigation strategies: pre-processing (modifying data before model training including reweighting samples, modifying feature values, and adjusting class labels), model modification (building models that inherently resist bias while maintaining accuracy), and post-processing (changing model outputs after prediction). The section describes IBM's fairness assessment tools including Watson OpenScale for monitoring ML models for discrimination, AI Fairness 360 open-source toolkit with comprehensive fairness metrics and bias mitigation algorithms, and VerifyI for generating synthetic data to test bias. The section acknowledges challenges including privacy concerns limiting data sharing and the need for auto-AI techniques that consider fairness alongside accuracy.

IBM's AI Fairness 360 toolbox provides a comprehensive framework for auditing and improving AI fairness, containing over 30 metrics covering group fairness (selection rates, equal opportunity) and individual fairness (consistency), plus sample distortion metrics. Approximately 10 bias mitigation algorithms span pre-processing (data modification), in-processing (fairness-constrained training), and post-processing (prediction adjustment). The toolbox supports multiple datasets (COMPAS, German credit, Adult census income) and integrates with common machine learning workflows. A practical workflow involves loading data, specifying protected groups, computing baseline metrics, training models, applying mitigation algorithms, and comparing results to assess fairness-accuracy trade-offs.

The Algorithmic Justice League, founded by Dr. Joy Buolamwini, conducts research exposing algorithmic bias in major tech companies. Starting from an art project where her dark-skinned face wasn't detected until she wore a white mask, this work revealed gender bias and skin type bias in AI systems from IBM, Microsoft, Amazon, and other major companies. This research contributed to companies stopping sales to law enforcement and influenced regulatory discussions like the EU AI Act, demonstrating how grassroots research can drive systemic change in AI governance.

Algorithmic justice represents the continuation and extension of the civil rights movement into the digital age. The connection between racially biased technology deployed by law enforcement, absence of elected oversight, and harm to marginalized communities became apparent during the 2020 protests. This demonstrates how technological systems can perpetuate and mechanize systemic racism, requiring collective action for meaningful change.

Computer scientist Joy Balamini founded the Algorithmic Justice League, using her research to expose how AI systems fail communities of color. Across the country, activists are demanding bans on facial recognition technology—cities like San Francisco, Boston, and Portland have already prohibited police use of these systems. Civil rights organizations are pushing for algorithmic transparency laws that would force companies to reveal how their systems make decisions. A new generation of technologists, many from the very communities most affected, are building alternatives designed with equity at their core.

Virginia Eubanks mentions the Algorithmic Justice League as an organization working to help communities respond to algorithmic and technological harm. She notes there is growing work among social movement organizations to think about what 'algorithmic justice' might look like, though there is less work on imagining how these tools could become truly liberatory.

A hierarchy of knowledge exists in technology fields where mathematical and technical expertise is privileged over qualitative, social science, and lived experience perspectives. This hierarchy reproduces historical inequalities from mathematics into computer science and big tech companies. Women and people of color face systemic barriers despite their expertise. Meaningful change requires collaboration between civil society organizations, affected communities, and academic researchers. Organizations like the Algorithmic Justice League, Black in AI, and Data for Black Lives provide platforms for collective action. Key strategies include local legislation (cities banning facial recognition), supporting affected communities in challenging predatory practices, and creating frameworks connecting data rights to human rights.