Coded Bias: Racial Bias in Facial Recognition & AI

[00:00:00]this is democracy now democracynow.org the war and peace report I'm Amy Goodman with nermeen Shaikh and we're broadcasting from the Sundance Film Festival in Park City Utah from Park City TV where a new film looks at the racial and gender prejudice baked into artificial intelligence technology like facial recognition the film is called coded bias earlier this month Cambridge Massachusetts voted to ban facial recognition joining a growing number of cities in the u.s. including San Francisco that have outlawed the artificial intelligence software citing flawed technology a recent study found facial recognition identified african-american and Asian faces incorrectly ten to a hundred times more than white faces the study by the National Institute of Standards and Technology found a photo database used by law enforcement incorrectly identified Native Americans at the highest rates the danger of flawed artificial intelligence and it's increasing omnipresence in daily life is the focus of the new film coded bias the film begins with Joya bone and we knee a researcher at the MIT Media Lab who discovers that most facial recognition software does not recognize darker skinned or female faces when she has to wear a white mask to be recognized by a robot she herself is programming she goes on to reveal that artificial intelligence is not in fact a neutral scientific tool but instead reflects the biases and inequalities of wider society this is joy Ballymena testifying before Congress in May I'm an algorithmic bias researcher based at MIT and I've conducted studies that show some of the largest recorded racial and skin-tight biases in AI system sold by companies like IBM Microsoft and Amazon you've already heard facial recognition and related technologies have some flaws in one test I ran Amazon recognition even failed on the face of Oprah Winfrey labeling her mail personally I've had to resort - literally wearing a white mass to have my face detected by some of this technology coding and white face is the last thing I expected to be doing at MIT an American at the center of innovation now given the use of this technology for mass surveillance not having my face detected could be seen as a benefit but besides being employed for dispensing toilet paper in China the technology is being used to track Weger muslim minorities beyond being abused there are many ways for this technology to fail among the most pressing are misidentifications that can lead to false arrest and accusations mistaken identity is more than an inconvenience and can lead to grave consequences that's joy balloon weenie who now joins us here in Park City at the Sundance Film Festival along with Shalini Kant aya the director of the film coded bias that's premiered here at the film festival we welcome you both to Democracy Now so take it from there joy I mean how did you end up testifying before Congress and take us on your journey from MIT discovering that your face is one that would be recognized so many fewer times when artificial intelligence technology is used than others I mean maybe that's protection who knows absolutely so my journey started as a grad student I was working on an art project that used face detection technology and I found that it didn't detect my face that well until I put on a white mask and so it was that white mask experience that led to questioning well how do computers see in the first place how's artificial intelligence being used and if my face isn't being detected in this context is it just me or other people can you also go step back what even does artificial intelligence mean what is AI mean sure so AI is about giving machines what we perceive to be somewhat intelligent from a human perspective so this can be around perceiving the world so computer vision giving computers eyes it can be voice recognition it can also be about communication to think about chatbots writer think about talking to Siri or Alexa and then another component to artificial intelligence is about discernment or making judgments and this can become really dangerous if you're deciding how risky somebody is or if they should be hired or fired because these decisions can impact people's lives in a material way can you talk about the origins of artificial intelligence you go over it a bit in the film coded bias oh yes and Shalini does a great job of really taking it all the way back to our Dartmouth where you had a group of who I sectionally call pale males coming together to decide what intelligence might look like and here you're saying if you could play chess well that's something that looks like intelligence the thing also about artificial intelligence is what it is changes so as machines get better at specific kinds of tasks you might say oh that that's not truly intelligent so it's a moving line so Shalini why don't you talk about how you came up with the idea for coded bias joy a central figure of course of this film and take the history further well basically I was sort of like a science fiction fanatic and so I like reading about technology and imagining the future and I think so much of what we think about artificial intelligence comes from science fiction it's sort of the stuff of Blade Runner and the Terminator and then when I started sort of reading and listening to TED Talks by joy and another mathematician named Cathy O'Neil other women like Meredith Broussard and saying up to fakey I realized that artificial intelligence was something entirely different in the now it was becoming a gatekeeper making automated decisions about who gets hired who gets healthcare and who gets into college and when I discovered Joy's work I was just captivated by this young woman who was disrupting the disruptors so let's go to a clip from your remarkable film coded bias this shows police in London stopping a young black teen this young black kids in school uniform got stopped as a result of a match took him down that Street just to one side like very thoroughly searched him it was all plainclothes officers as well it's four plainclothes officers who stopped him [Music] fingerprinted him after valve like maybe 10-15 minutes of searching and checking these details in from the Pentagon they came back and said that's not him giveaways campaigning organization they're campaigning against facial recognition technology we're campaigning against a ship father what shareware human rights organization campaigning against this technology here today and then you've just been stopped because of that but they misidentified you key details he was a bit shaken his friends were there they couldn't believe what had happened to them you'd be misidentified by their systems and they've stopped you and you start as justification to stop and search you but this is an innocent young fourteen year old child who's been stopped by the police as a result of a facial recognition misidentification so that's a clip from code bias Joey bellum weeny explained further what took place here the misidentification the identification some might perversely say it's better for this technology to fail so the people can't be identified but this is the opposite case absolutely so you were saying earlier maybe not being identified as a good thing but then there are the misidentifications that have a real world impact so in the clip and in the film you actually see the work of Big Brother watch UK and in this particular scenario Big Brother watch UK was able to track what was going on in London and one of the things they showed it in their study face off was he had false positive match rates of over 90 percent so you see this one example here but they also had reports where more than 2400 innocent people were mismatched so it's not just a case of oh you're not detected that might be sometimes but you could be misidentified as somebody you're not and the consequences can be great and we're playing this clip at a time when the New York Times reports London's Police Department London's Police Department said it would begin using facial recognition to spot criminal suspects with video cameras as they walk the streets adopting a level of surveillance that is rare outside China the technology London deploying goes beyond many of the facial recognition systems used elsewhere which match a photo against a database the new technology uses software that can immediately identify people on a police watch list as soon as they're filmed on a video camera joining and I think you might need to say attempt to identify because often times the claims that are made about these technologies don't necessarily match up to the reality earlier you spoke about the National Institute of Standards and technology's study they studied more than 189 algorithms from 99 different companies and so this is the majority of the facial recognition technology that's out there racial bias gender bias age bias as well so if you have a face you have a place in this conversation and we all need to be concerned so I think it's highly irresponsible to deploy technologies that we already know have significant flaws that we already know can be abused it's common sense to place a moratorium until we're at a better place well surely another place that you profile in the documentary is China and you speak to this woman at some length so a couple of questions first how did you get access and your response to the fact that she actually supported the credit what is it the social credit system absolutely would explain what that is how it works there and what your senses of the kind of support that this system has in China and then joy along the same lines as what you were talking about earlier in places like China where the artificial intelligence and facial recognition the technology is developed there is there a similar bias and if so what what is it but first Shalini well I got accessive through local production company in China and I feel that this woman kind of gave us insight into this social credit system that is coming up in China to sort of where they're using facial recognition in tandem with a social credit system so if you basically they're tracking you they're watching you they're surveilling you and they're scoring you and not only what you do impacts your score but what your friends do impact your score and this young woman who I am who is featured in the film says that you know in fact we don't have to trust our own senses anymore that we can rely on this sort of social credit score to actually have integrity and who we trust and who we don't trust and I think in the film you know we sort of want to think oh that that's sort of a galaxy far far away from the US but in the making of this film I see all kinds of parallels of that type of scoring that's happening here in the US and in other places around the world what how you see that it's comfortable or it could be well as as Amy Webb says so poignant Lee and in the film we're all being scored all the time from our uber scores to our Facebook Likes all of that information is being tracked and analyzed all of the time and so we're all being rated all of the time and so though that kind of tracking can impact how much we pay for insurance what kind of opportunities are shown to us online and so very much it becomes sort of an algorithmic algorithmic determinism enjoy the question of how are the systems working in China in our first study called gender shades we looked at IBM Microsoft but we also looked at face plus plus billion dollar tech startup in China and we found a similar racial bias and gender bias but overall when they've done Studies on AI systems developed in China they tend to work better on Chinese faces right and those developed in Western nations tend to work better on Western faces as well one thing I did want to bring up related to China and data collection is this data colonialism that we're starting to see so you have reports of Chinese companies going to African nations providing facial recognition or surveillance technologies in exchange for something very precious the biometric data of the citizens so now parallel to what we had with the slave trade right where you're extracting bodies now you're extracting digital bodies in service of a global trade because even when you talk about what's going on in London they're using technology from a company called NEC that's based in Japan and so you have to really think about the global context for how these technologies spread around the world and just to add to that China has unfettered access to data it has now been mandated that if you want to access the Internet in China you must submit to facial recognition so that has that that is the basis for which they're building this kind of scoring system as we wrap up what about regulation these algorithms are impacting all of us in the most in our civil rights and we need legislation we need Algar we need meaningful legislation around algorithms and the explanation of algorithms in just 20 seconds joy for us non-scientists yes so algorithms that are essentially processes that are meant to come to give or solve a particular task so when we talk about AI we're talking about systems that can perceive the world that can communicate and most importantly make determinations and these determinations impact our lives well we want to thank you so much for being with us joy of all my knee is researcher at the MIT Media Lab and founder of the algorithmic justice league we're gonna link to her speeches and her congressional testimony at democracynow.org and Shalini Khan taya is director of the new film that's just premiered here at the Sundance Film Festival called coded bias and that does it for our broadcast on Friday at 2 p.m.

[00:15:57]and I'll be speaking here in Park City Utah at the Museum right next to Dolly's bookstore about impeachment and elections next Tuesday February 4th we'll be in Washington DC just before the State of the Union I'll be interviewing Lonnie bunch founding director of the Smithsonian's National Museum of african-american history and culture at six o'clock at Busboys and poets in Washington DC and next Friday February 7th at noon nermeen Shaikh will be moderating a panel with a squad at Howard University Alexandre Castillo Cortese receded to leave Ilhan Omar and Ayanna Presley the for Congress members will be with nermeen at Howard University at noon February 7th you can go to all details at democracynow.org also Democracy Now will be broadcasting the Senate impeachment trial at democracynow.org I'm Amy Goodman with nermeen Shaikh you