The Risks of Police Facial Recognition Technology

Algorithmic auditing adapts traditional social science auditing methods to study opaque algorithmic systems. Researchers systematically vary inputs and analyze outputs to detect bias. Joy Buolamwini's facial recognition audit revealed systems performed worse on darker-skinned female faces. Danae Metaxa's Google image search audit found systematic underrepresentation of women and people of color (7-15% difference from Bureau of Labor Statistics). Beyond representation, research shows algorithmic content shapes user perception—increasing women in search results made occupations seem more inclusive but paradoxically decreased overall interest, revealing occupational feminization dynamics. Algorithmic bias originates from upstream factors: training data reflecting historical discrimination, designer biases, and broader social systems generating input data. Audits serve multiple purposes including internal evaluation, journalistic investigation, and policy analysis. However, audits have limitations—they only test specific metrics and may inadvertently legitimize harmful systems by showing adequate performance.

There are three main approaches to mitigating algorithmic bias in machine learning pipelines: (1) Pre-processing - sanitizing raw data before training to remove existing biases; (2) In-processing - incorporating fairness constraints directly into the training algorithm; (3) Post-processing - modifying model outputs after training to achieve fairness. Each approach addresses bias at different stages of the pipeline, and the choice depends on where the source of bias is identified.

Algorithmic auditing is a three-phase methodology for ensuring ethical AI development: Phase 1 involves understanding how social problems are converted into data processing problems, including identifying what data is needed and recognizing historical biases in training data; Phase 2 focuses on technical auditing to observe how algorithms impact different groups; Phase 3 addresses the integration of algorithmic decisions into organizational processes and human oversight. This approach emphasizes that ethical principles like transparency, non-discrimination, and equity must be translated into concrete technical specifications, and that effective auditing requires interdisciplinary collaboration between technical and social science perspectives to address the complex social dynamics encoded in algorithms.

This final section examines human-centered challenges and technical methodologies in auditing complex algorithmic systems. Platform operations involve rapid human decision-making under time pressure, creating risks of implicit bias similar to implicit bias test conditions. Content moderators and users alike make snap judgments that may introduce errors when verifying algorithmic decisions or when platforms rely on rapid behavioral signals. Platforms often misrepresent their functionality—claiming to show users what they want when they actually show content similar to historical engagement patterns. Children present particular challenges as their developing brains respond differently to algorithmic stimuli than adult brains. Technical auditing faces fundamental challenges: low explainability, lack of direct system access, and algorithmic changes over time. Black box auditing focuses on downstream behavioral effects rather than internal mechanics. Simulation tools and procedural audits examining engineering documentation offer alternatives. Reproducibility remains difficult due to changing algorithms and non-standardized data access. Solutions require standardized datasets, simulation capabilities, and recognition that procedural audits examining decision-making processes can complement technical evaluations.

This extensive section covers the technical aspects of bias in algorithmic ranking systems, including distinguishing between societal biases (mismatches between ideal world expectations and reality) and statistical biases (how information is encoded in systems). Key bias types include data bias (unbalanced training data), popularity bias (rich-get-richer effect), and presentation bias (serial position effects). The section details measurement approaches including the recommendation gap metric for demographic bias and Delta Matrix metrics for popularity bias. It presents three categories of bias mitigation: pre-processing (data rebalancing), algorithmic integration (adversarial training), and post-processing (re-ranking). The section also explores transparency as a crucial requirement, categorized into algorithmic transparency, decomposability, and understandability. It discusses explainability goals and the importance of algorithmic auditing as a process where third-party experts test systems for bias and discriminatory behavior.

An advanced civilization designing a kinetic weapon wouldn't make it obviously artificial. They'd study natural comets extensively, their composition, their behavior, their statistics, and then build their weapon to mimic those properties as closely as possible. The goal is to blend in, to pass every test we can throw at it. In that scenario, we'd expect to see something that's almost natural—something that fits most of the expected patterns but has subtle anomalies. Anomalies small enough to be plausibly deniable, but collectively suggesting something more. This is the adversarial machine learning problem: if an opponent knows what your detection algorithms are looking for, they can design their attack to evade detection. It's the same arms race we see in computer security—hackers study antivirus software to design viruses that evade detection, antivirus companies update their algorithms, hackers adapt, endless cycle.

Adversarial fashion is clothing and accessories designed to trick facial recognition software by confusing the neural networks that categorize objects, using exaggerated features like oversized ears and noses to make machines uncertain whether they're seeing a human, thereby protecting privacy in an era of mass surveillance.

Hitmen employ sophisticated counter-surveillance techniques to evade law enforcement. When FBI agents track a hitman's location, the hitman can quickly neutralize threats by triggering hidden explosives. Hitmen hide weapons in ice cubes, place explosive devices in carpeting, and use remote-controlled explosives to destroy pursuers. When trapped in elevators, hitmen use the elevator's automatic descent to their advantage, demonstrating environmental awareness and preparation essential for hitman survival.

An inherent challenge in AI detection systems is the adversarial nature of the problem: as detection algorithms improve, attackers can develop better countermeasures. When a detection system achieves high accuracy (such as 99%), attackers can generate thousands of samples, identify which ones evade detection, and refine their techniques accordingly. This creates an ongoing arms race between detection capabilities and evasion strategies. The V7 Labs AI detector represents one side of this dynamic, while attackers developing more sophisticated AI-generated content represent the opposing force. This cycle demonstrates why continuous improvement and adaptation are essential in security technologies.

Modern counter-surveillance includes infrared camouflage glasses that appear as glowing blind spots to cameras, exploiting wavelength differences. Search dogs can detect individuals by unique sweat chemical signatures, with just one gram sufficient to locate someone on a ten-story rooftop or 15 feet beneath concrete. Anti-drone weapons use electromagnetic radiation to disrupt communications without precise targeting. Facial recognition technology measures distances between facial landmarks (nose, eyes, mouth) to create numerical biometric codes compared against databases. These technologies represent the ongoing arms race between surveillance capabilities and evasion methods.

Predictive policing creates dangerous feedback loops that reinforce historical policing practices. Areas with heavy police presence show more crime simply because more police are present to discover it. PredPol analysis showed algorithms repeatedly targeting the same blocks, clustering predictions in predominantly non-white, lower-income neighborhoods. Every prediction changes police behavior, which gets folded back into future predictions, creating self-fulfilling prophecies. This demonstrates how well-intentioned data-driven approaches can perpetuate and intensify existing inequalities.

Predictive policing creates dangerous feedback loops because it relies on historical police data including arrests and calls for service, which doesn't just represent actual crime but reflects where police have concentrated enforcement. When police focus patrols in certain neighborhoods, those areas generate more police contact, leading to more stops, arrests, and public order violations, which then becomes part of the training data for new models. This amplifies historical policing patterns rather than predicting true crime rates. Case studies like Pasco County demonstrate how this manifests: individuals appear on watch lists, face repeated surveillance even without committing crimes, and petty offenses become part of the algorithmic training data, legitimizing discriminatory practices through technological means.

Predictive policing uses algorithms to predict crime hotspots or high-risk individuals based on historical data. However, historical crime data often reflects historical policing patterns, creating feedback loops where algorithms prove neighborhoods are high-risk only because police have always looked hardest there. Core accountability questions include who audits algorithms, who tests for bias, and who can challenge outcomes when innocent people are harmed by algorithmic decisions.

Predictive policing programs use historical crime data to predict future crime locations, but results depend entirely on which crimes are included in the data. Including minor crimes versus only serious crimes produces different geographic predictions. This creates feedback loops where increased police presence generates more arrests, reinforcing predictions and justifying continued presence. The system can direct resources to areas with frequent minor crimes while overlooking areas with fewer but more serious crimes, potentially perpetuating existing social inequalities and geographic biases.

Predictive policing algorithms learn patterns in police records, not actual crime occurrence. Since police records are biased, the algorithm learns patterns in where crimes are detected rather than where crimes actually occur. Analysis of Oakland drug crime data shows most crimes concentrated in specific areas (West Oakland and International Boulevard), while public health data reveals drug use is actually more uniform across the city. These over-policed areas are primarily occupied by African American and Hispanic communities. When predictive policing algorithms are applied to biased data, they reinforce existing bias rather than correcting it. Black people are about twice as likely as white people to reside in locations receiving predictive policing.