Data Ethics & AI Justice with Renée Cummings | GVSU Talk

[00:00:00]thank you so much everybody for joining us this evening so I'm Aaron your van osterhout and I serve as a research manager at the community data and research lab at the Johnson Center for philanthropy here at Grand Valley I also have the honor of serving as the co-director of grand Valley's Own Prison education program and that's the Bellamy Creek Campus and I can assure you that few people's lives are as profoundly shaped by our society's belief in the supposed objectivity of data than those people who find themselves enmeshed in the criminal justice system it is my distinct honor and pleasure to introduce someone who will shake that belief this evening assistant professor of the practice in data science and the school of data science at the University of Virginia Renee Cummings joined the school in 2020 as the School's First Data activist in Residence she is a criminologist criminal psychologist artificial intelligence ethicist therapeutic jurisprudence specialist and urban technologists her areas of research interests include artificial intelligence political science and criminology she studies the impact of artificial intelligence on criminal justice specifically in communities of color and incarcerated populations in her work as a criminologist Cummings founded criminal justice intelligence Incorporated which works with governments in the Caribbean to strengthen crime prevention incorporate new technologies in existing crime prevention strategies and develop behavior and Hardware strategies that Interlink to reduce vulnerabilities and tailor more proactive approaches to Crime Control and crime reduction Cummings also founded Urban Ai and is an East Coast Regional leader for women in AI ethics in her time in New York City Cummings brought her expertise to train police officers and law enforcement agents to decrease homicides and gun and gang violence Cummings specializes in implicit bias AI ethics and best practice criminal justice she holds a master of Masters of Science and Rehabilitation science and substance abuse counseling from Hunter College a master of arts in criminal justice from John Jay College and a master's certificate in terrorism studies Cummings designed her own bachelor's degree at Hunter College in political science philosophy media studies and creative writing Cummings is also a community scholar in artificial intelligence and Criminal Justice at Columbia University it's my pleasure again to welcome Renee Cummings to the stage thank you so much thank you so much for that amazing introduction uh let me say it is indeed an honor and a pleasure and my distinct uh privilege to be with you uh this evening uh we are going to have a conversation about data algorithms artificial intelligence and Justice and I hope somewhere along that line in that conversation we realize the extraordinary amount of work that we've got to do as a collective when it comes to ensuring our technology that's being deployed new and emerging Technologies are responsible trustworthy and always ethical so let's begin this conversation so the conversation is disrupting data Injustice and let's just start by thinking about something we have seen the extraordinary impact of data science on society data science is considered probably the career of the now everyone wants to be a data scientist and we're still trying to Define what is data science what are the roles and responsibilities of the data scientists but we do know that we have experienced individually and collectively the positive impacts of data science and there are many of them but we also know that what we are realizing is as we experience what is positive and what is transformative about this amazing technology that we're designing new and emerging Technologies with data we're realizing that there are many consequences and what we're also realizing is that while we are solving some of the world's greatest challenges with data we are also creating new ethical challenges that demand our real-time attention so the Big Challenge how do we harness the power of data while mitigating the risks and reducing the harms that's what we're trying to do with data science as we try to solve some of these extraordinary challenges that we have as a world a society and a globe The Dilemma that we have is that data is as powerful as it is vulnerable it is probably the most powerful thing commodity asset that we have to use but as it is powerful it is extremely vulnerable vulnerable to attack to breaches to Bad actors and these are some of the reasons why we've got to think about the ways in which we need to do technology that is Justice oriented equitable so diverse and inclusive so we know the data is profit it is big money and we know that there are five big tech companies that own us that own our data and we also know if we've been using social media if we've been on the internet if we've been doing any kind of research watching any kind of media or doing any type of online shopping once we have stared down a computer and clicked we have data points and those data points are being used to create these data profiles about us and what none of us actually know is what does our digital twin actually look like and that's something we need to think about so what we're seeing with data data is knowledge data is intelligence and with that intelligence and that data Acumen what we are doing is creating more access in places that we have never been in systems and design and and process these and procedures that we have not even thought about we are using data to now create and to stretch the imagination with access and that knowledge and that intelligence we are creating extraordinary opportunities and these opportunities are redesigning the business model and reimagining that business model because what we are seeing is that every sector every system every discipline every industry is being impacted by data science it's being remodeled reimagined redefine repositioned because of data and what data allows are many of the individuals who actually own that data who are designing with that data who are deploying these systems would be leadership because that's what data is about who has the data is allowed to lead and to lead in the ways in which we are thinking and the ways in which we're interacting with each other and the kinds of systems and processes and procedures we're developing so data is also about development so there's some great things about data but we know that data is promise and if it is we are searching for this ideal we've got to deal with the privilege and we've got to deal with the Prejudice because these are the dangerous things about data and the ways in which we are using data and the power of that data so the politics and there's extraordinary amount of politics around data not only a commodity it's not only about economics but data is cultural data is also political and we're seeing data being used in the realm of global dominance and we're also seeing it being used to expand to expand the ways in which Industries are expanding the ways in which governments are expanding and the ways in which tech companies are expanding in ways in which governments cannot even expand and we're seeing the monetization of our data but as we see the monetization of our data what we're also seeing is the weaponization of that data and particular communities being more vulnerable to that weaponization and there are two things that I always say is that all data are not created equally and different communities experience data differently and that's when I say weaponization what I'm thinking of there so if we are to do data science right we have got to think about the ways in which we have historically been collecting data the kind of analysis around that data the quality of the data as well how we are using that data to visualize the kinds of challenges we have uh the ways in which we're presenting that data and the ways in which we are sharing that data and there's some things that I've realized working as a data activist and an AI ethicist and working as a criminologist is that many times we do not think about data and data's memory and that data has an extraordinary memory beyond the numbers beyond the statistics beyond the computation it has a data has an extraordinary memory when it comes to oftentimes replicating trauma and Trauma in particular communities so we know when we think about data it's about decision making accuracy that's what everyone wants that's what data Acumen is it's about how do we make the most accurate decisions in real time to really exploit the kinds of systems that we're creating but where there is this desire for excellence through efficiency and Effectiveness and oftentimes expediency there's this challenge of exclusion because what we are also seeing is the data has an extraordinary power to exclude because what we decide to include in our analysis of data it means that there's some things that we have also decided to exclude and those are some of the challenges we are seeing and that we need to think about so we know this decision making is about knowledge it's about opportunity and it's about resources but it's also about something even more critical it's about legacies and what we are seeing is that there are many communities that are now being challenged to uh get the kinds of resources that are required to create sustainable and resilient legacies because they no longer have access to the resources because we're using data to decide who has access to the resources so we're using data to decide who gets a loan who gets particular housing who gets a mortgage who is determined and deemed credit worldly who was deemed uncredit worthy we're thinking about data as well in the criminal justice system and the ways in which historic data are really re -administering things that we thought we had in check so when we are thinking about data science and coming from a critical data science perspective we've got to start to think about how do we really interrogate our data sets how do we investigate how do we get behind the data and really understand the history the memory the kinds of experiences that particular communities have had that are attached to those data sets because what we have realized but many of the historic data sets that we have continued to use to build systems would be that many of them are deeply destructive because they continue to replicate and repeat past prejudices as well as what we have seen is that we're using new and emerging Technologies and we're using those Technologies with historic data with old data and what we're doing is automating the status quo what we are also doing as we have seen particularly in criminal justice is that we're codifying bias and bigotry and those are things that we don't want to do so when we think about those hidden effects of data we are thinking about the racial disparities we are thinking about the economic disparities marginalization and of course systemic racism all of these baked into our data sets so just things that I want you to think about as we move into the conversation as well about algorithms and AI so some of the hidden effects of historical data many of you may have read this study that was revealed investigative research slack study uh that was uh executed by propublica and what Republica did was look at the recidivism risk assessment tools that were being used by the criminal justice system and what they realized is because we were using historic data historic data was over estimating the kinds of risks of black and brown men in the criminal justice system and creating what we call these zombie prediction saying that particular groups were going to reoffend and that we needed to deny particular groups bail because they were going to reoffend and when we drilled into the data we realized it was because of those disparities that were baked into those data sets those hidden effects of historical data that we really don't think about implicit bias you know unconscious bias and for us to really understand the kind of responsibility that we have as technologists data scientists sociologists economists designers design thinkers uh we have got to realize that we have got to be extremely diligent because some of the things that we've seen these artificial assumptions would be that there is this view that technology is neutral because it is math because it is science we just believe that it is right and what we have come to see is that no technology is neutral we have got to continuously question the objectivity of Technology because of the opaque nature of many of our Data Systems many of our algorithmic systems are data driven and algorithmic decision-making systems because they continue to perpetuate the biases and the disparities so we've got to be diligent and that diligence means as we've always got to be thinking of how do we detect how do we mitigate how do we manage how do we monitor these algorithmic and data-driven systems that we're using and how do we use that feedback and that information to continuously bring an ethical approach to the ways in which we are doing data science and doing artificial intelligence so there's also the question of Duty of care and if we are Designing Technology for each other for Society for us to use we have a responsibility as technologists and data scientists and anyone using data Healthcare professionals to really think about how we're doing that so if we are to disrupt the injustices it means that we need to take an ethical approach an ethical approach one that is mature responsible building trustworthy technology it means that we've continuously got to think about oversight we've got to think about compliance we've got to think about regulation and governance and we have got to bring that Justice oriented and trauma-informed perspective to the ways in which we are thinking and the ways in which we are doing data science and it's also about you process because what we have seen with data-driven systems and algorithmic uh decision-making systems would be that many times we're using data and creating systems to make decisions and as these systems continue to make decisions they are leaving us out of the decision-making process so we see questions of agency and autonomy and equity and fairness and self-determination being threatened because of the fact that many of the systems we are designing we're designing without approach an approach that really understands how critical due process is in the design of new and emerging technology as well as this deployment and adoption so it brings us to the question of diversity and we have got to think about the diversity of harms being deployed as well with this new and emerging technology and we've got to think about the diversity in the cognitive injustices that many communities have felt and experienced because of data we've got to think about accessibility and whether or not we are truly making technology accessible we've got to think about representation and how critical voice and visibility are to the ways in which we represent certain groups and how do we do that through multi-stakeholder engagement it is critical because it is about Community engaged research it's about Community engaged design it's about really bringing the community into the process to inspire the design and it's about the interdisciplinary imagination because we realize in data science that data science needs all of us needs all disciplines so we need the sociologists the psychologists the criminologists the uh you know just about everyone the Educators the every industry every discipline needs to be a part of data science and needs to be a part of artificial intelligence because we're realizing diversity in thinking and in the ways in which we are doing data science and in the kinds of conversations that we're having uh to create the requisite level of intellectual confrontation that's required to build an ethical approach to data science so diversity also very critical the challenges that we continue as we design as we develop and as we deploy to have to deal with discrimination and I come from a background of data Justice and today I had also a very interesting conversation with design students about design Justice and how critical design Justice is to data science and critical data science and artificial intelligence so racial justice social justice climate Justice these are all critical questions around the ways in which we need to do data ethically because it comes down to this democracy if data science and using data is about data and decision making accuracy it's also about democracy and what we're seeing with data science is that while it has the potential to do things that are extraordinary and think that are transformative it also has the potential to disenfranchise and we have experienced disinformation and we have experienced deep fakes what we don't want to experience would be more deep fakes because that is an experience we are not ready for as a society as yet and that also speaks to data literacy and the lack of data literacy and AI literacy and the lack of AI literacy and Futures literacy and of course the lack of Futures literacy because this is the language in which we are now communicating data and algorithms and a requisite level of literacy is also required to communicate with these languages because we are seeing that an unethical approach to data science a lack of a critical data science perspective actually undermining and dismantling our Democratic rights so what we don't want is to use data to disempower we don't want to undermine agency and autonomy we don't want to compromise self-determination and self-determination is also so critical among our first people than our first nation and our indigenous communities who are committed to thinking about data governance in new ways who are committed to really bringing a robust and rigorous approach to data ethics through data stewardship who are having conversations about indigenous data and the indigenous imagination and how do we include that in the conversation because we are seeing the data that is deployed in unethical ways that is used to build Technologies in unethical ways that are building unethical algorithms have the power to encroach in our civil liberties our civil rights and of course our human rights so for us to democratize this technology which means bringing all of us into the fold bringing all groups into the conversation we have got to find ways in which we could demystify we can decolonize and we could really emancipate the data from its historic past a past of pain a past of travel a past of injustices a passive and quality inequalities and inequities because we're seeing the world moving in the direction of techno-solutionism and techno-solutionism is simply this belief that every challenge needs a technological solution and it doesn't and we have realized that and techno determinism where we have a very deterministic way of now looking at the world and believing technology is going to be our savior and that's not going to happen so we've got to think about how do we demystify how do we decolonize how do we emancipate for us to democratize the ways in which we are using data to ensure that we always understand why it needs to be done ethically and it brings us to data trauma and the fact that when I mention to you that different communities experience data differently we know that history and we understand the pain of that history and we understand if we are supposed to use data to build new and emerging Technologies and to disrupt the injustices of data then we have got to understand the trauma attached to data for many communities so we've got to always ensure that when we do data we are doing it in a way that is culturally aware appropriate competent sensitive and responsive because this is what we have seen with data trauma data bias data discrimination data profiling data weaponization and data victimization all critical and I come from that space of Criminal Justice and criminology and policing and ways in which we could do policing in more Equitable and diverse and inclusive ways and we could bring communities into that conversation because we know that communities and law enforcement are co-producers in keeping a crime rate down in any space so we have got to think about the trauma attached to that and we have got to understand if we are using data then we have got to use it in a critical way which includes bringing a very responsible and mature approach to the ways in which we are thinking about technology so of course there are the ethical risks and the ethical risks around data would be privacy accountability transparency explainability of course fairness and non-discrimination and security and safety so we're coming up with ways in which we can deal with those ethical risks because they have major ethical challenges for organizations many organizations have deployed ethical Frameworks many organizations are committed and governments committed to doing data ethically most recently the White House released the blueprint for an AI Bill of Rights which looks at the ways in which we need to think about Ai and algorithms and how they are impact in certain communities and bring a safety approach to it so we've got to think about rights and we've got to think about risks of course we know the gdpr the general data protection regulation coming out of the EU and at the moment the EU having conversations about the euai ACT which is supposed to be enacted very soon so we say Do no harm and that's the big one but we're trying and we believe that if we are to do data ethically and if we're to do algorithms and AI ethically that we should do the least amount of harm possible so we've got to create this rigorous and robust guard rail that's required to protect the ways in which we are interacting with data because it's about public confidence and it's about public trust when you combine public confidence and public trust that's where you get legitimacy that space of legitimacy in which we are using these Technologies and it comes back to being ethically Vigilant and ethical vigilance is critical because it means eternally as you are using data as you're using data to design whatever it is that you want to design or respond to whatever those challenges are you are understanding that ethical responsibility that you have because it comes down to data rights and data rights is probably the civil rights movement of the now because these are the conversations we need to start having if we're not having them yet and these are also kitchen table conversations because we need to have them with everyone in the family because everyone is using data all communities and each and every one of us in this space represents many communities so this is why this conversation is so critical because we're seeing data the unethical use of data having the extraordinary potential to do great and transformative things but also to impact our civil rights and our human rights in a of course our civil liberties and we spoke about that AI Bill of Rights and the EU act in the gdpr and how these approaches to governance and and Regulation and compliance are assisting Us in building those robust guard rails that are required so data rights and we've got to think about that so now we come to algorithms and this is going to be a quick and you know real-time crash course in How We Do Ai and algorithms so we know that it's a formula that provides an instruction to a computer to perform a very specific task some people say it's like a recipe so you put certain ingredients in there you you know in the computational way and then the computer is able to deliver what looks like a very human-like uh kind of response to the things that we want so there are these algorithms and these algorithms are continuously being designed and we are using them and we are using them so much but so many of us don't know we're using them and that's why one of the things that really interests me with the EU AI Act is the fact that the ACT says and they're looking at maybe it's time for us to let people know they're using algorithms so when you apply for something alone or when you apply for a mortgage or whatever it is you're using on the the internet you know so much you know that you are able to know that this is an algorithm you know so many of us use let's say uh Amazon we're shopping um how many of us have even asked ourselves what is Amazon choice you know we see Amazon choice we oftentimes click in it but what is really the choice you know whose choice is that and what's directing us to that choice so we've got to think about that so these algorithms and you cannot have algorithms without data you cannot have ai without data because data is what we use to create these algorithms and we have been seeing that because most people don't know they're using algorithms because there's nothing that says an algorithm is in use here there is no warning that these algorithms continue to make decisions about us and for us without us even knowing so when I spoke about agency and autonomy and self-determination these are the challenges that algorithms are creating because what they are doing is undermining our identity undermining what it means sometimes and oftentimes to be a citizen so the question is how can we maximize the impact of algorithm because they can have very positive and Powerful impact on society and mitigate those adverse effects and that's what we're trying to do with AI ethicists and data activists and and data ethicists and individuals working in the data science space so one of the things that I always say when we come to defining AI it is still undefined that's the only definition it continues and remains to be undefined but we know that these tasks are being done in a way that replicates humans human behavior we know it requires an extraordinary amount of data we know that it's life-changing the kinds of decisions we see algorithms making we also know that algorithms have the opportunity to create extraordinary legacies but algorithms also have the opportunity to destroy legacies and that's what we've got to think about how do we build sustainable and resilient legacies for all and how do we mitigate those risks of algorithms undermining those legacies so this is Joy below meaning Dr joy bulomini and uh she's known as really a main thinker main researcher within the space of algorithmic justice and algorithmic Injustice and what she realized while doing a project for MIT before she graduated with her doctorate was that she was trying to Repro Zend her face using image recognition technology and because of her dark skin that was impossible she had to put a white mask on and she realized at that moment facial recognition technology was definitely not a technology that was being used effectively for black and brown people and particular women as well or anyone who identified as a woman so she said that AI systems shaped by the priorities and prejudices conscious and unconscious of the people who designed them so we've got to check ourselves as designers as individuals working in that design space because algorithms have created a new space for Injustice and she introduced what she called The Pale male bias because up to 77.4 percent of subjects in mainstream data sets are male up to 94.6 are lighter skin males and what this is creating is the dysfunctional dangerous space for individuals of dark skin black and brown communities so facial recognition and this gentleman right here Robert Williams is with his family but he was arrested because the police said they saw him in a mall shoplifting when he was not in that space but they used facial recognition technology that was deployed in that space and somehow it produced an image of a man they said that looked just like him so he spent 15 hours incarcerated before a police officer decided to look at his ID and realize that the two pictures were not a match but think about the trauma that his children went through because he was arrested at home in front of his family of course in a manner that was not very polite or very civilized and the kinds of extraordinary impacts of that he lost his job in the process because he was on his way to work and he couldn't go to work so we're using technology and we're deploying technology in a very capricious and a Cavalier Manner and we have got to think about that so we know there are many conversations at the moment around facial technology some cities of decide to ban it the EU was saying it's way too high risk to deploy and we've got to think about not using it so they also want to ban it but we're seeing the positive effects of facial recognition technology and Finance in healthcare so these are some of the questions and some of the challenges and of course predictive policing where we're using data to create predictive systems that are supposed to reduce crime and criminality but what we continue to do would be to replicate those old biases with new technology apology and not come up with new Criminal Justice strategies but come up with strategies that continue to victimize to track to trace to terrorize certain communities and at the moment at the University of Virginia I'm working with some data scientists my own team what I call my mini lab and we have designed what is called a digital Force index and that gives you a digital Force score because what we have done is call it all the surveillance Technologies private and public that are being deployed in this country and coming up with a score so when you go into a particular Community you could know whether or not the Technologies being deployed the surveillance Tech being deployed whether they're high whether it's medium whether it's low and how this technology could impact you and your future because there are things called geofencing in geolocation warrants and we're realizing now just with your phone and the data that if your data turns up at a particular place where police may be doing an investigation you can become a person of interest to the police in the blink of an eye you're now part of an investigation so we've got to think about predictive policing as well so if we are to do a responsible and trustworthy AI then we've got to include everyone in the conversation if we are to disrupt data injustices we've got to bring all citizens in there Civil Society has an extraordinary impact stakeholders are critical because it's about public engagement it's about public information it's about public education to ensure we are doing technology in ways which are responsible and of course always trustworthy it's also about all organizations Academia working with governments working with Regulators to really create that space that is responsible and trustworthy because it is so critical and when we think about data and we think about AI we have always got to think about Justice Equity diversity inclusion and this is a campaign uh for women in AI ethics and what we try to do is show the future of AI well that's me there um using a variety of women cultures sexuality to show that AI looks like all of us because that's how we democratize that's how we demystify that's how we bring an understanding to the ways in which we're doing technology and that's how we become inclusive because we want to bring everyone into that space so as powerful as data is as powerful as algorithms are and given the fact they have this extraordinary ability to do extraordinary things and to do really great things we've got to think about that vulnerability and that vulnerability that ability to be used in ways which are unethical have got to be very important to all of us because it is our data that's being used it's our data that's being monetized it's our data that's being weaponized and in as much as we need data to create the kinds of technologies that continue to transform our world our globe and our experiences we have got to be critical about those ways and we have got to bring an ethical approach and we have got to bring that ethically resilient approach and we have got to bring an approach that is sustainable an approach that makes sure that we are continuously Vigilant about the ways in which we're doing technology because we need it we need technology and we need data but we also need all of us to experience the benefits of that and we have got to be committed to this understanding that if we are doing technology if we're doing AI we're doing it for all of us we cannot continue to just do it for some of us because as we are using technology to remove barriers we are also erecting new barriers with Technologies so as an AI ethicist my focus is on fairness accountability transparency in particular auditability and looking at privacy and security and really thinking about ways in which we can do it so I spent a lot of time traveling internationally this was in Montreal and really having these kinds of conversations about how do we do data science ethically how do we do data AI right and as a data activist we've got to understand that each of us is a data activist each of us we have got to be responsible we've got to be on the ball when it comes to how our data is being used and we've got to think always about being Justice oriented and and Trauma involved and informed and ensuring that the Technologies we are creating are people powered Community inspired using data for change AI for good and I'm also the co-director of the public interest technology Network at the University of Virginia which is so critical and many uh universities are part of this network because we've realized that we've got to use technology in the public interest we've got to build technology with the public in mind we've got to bring the public into that conversation because this is so critical a conversation that we've got to understand the role that we play our responsibility for us to get data right and with that I want to thank you very much for this opportunity and always thanking you for the privilege of your time thank you so much [Applause] thank you for the presentations the question on your your slide on Do no harm that seems like a great place to start but is there even a good definition for that fantastic where would somebody start fantastic and I think we start with ourselves and we start with that understanding that we have got to bring that requisite level of vigilance and due diligence to the ways in which we're using data and to the ways in which our data is being used so we've got to really get involved and you get involved in that conversation wherever you are I always say wherever we sit wherever we stand we lead because we represent so many different communities so it's really about having those conversations it's about ensuring that a local government level at a national federal level that we're having these conversations we're seeing organizations moving in that direction but even in our schools in Academia in our classrooms at our kitchen tables we need to start talking about data because the challenges I say to you is yes I'm in the space and I'm seeing how technology is being you know developed at Breakneck speed but I'm also seeing how many different groups are being left out and I bring that commitment to legacies I want to see those Equitable legacies I want to see all groups really moving into the future as we continue to move in ways in which we can all experience the benefits and I always say to data scientist and to my students yes we may be as individual data scientists creating products and systems and tools but as a collective we are creating our future and we have got to think in that you know Direction what kind of future are we building and who are we including and who are we excluding so I say wherever you are just have that conversation and just be part of that conversation thank you um I really enjoyed your presentation thank you uh specifically the um how different cultures interpret data I thought that was really interesting um this has been fascinating and really really helpful on awareness what's something actionable that that I should do as a next step or you know what what's the next thing that we should all be trying to learn about so that we can help move this forward also thank you again I think the next step as I said is just getting involved if it is that you take a data science class or a data ethics class or it you create a conversation or you just get involved in the conversation ensure the conversation is happening wherever you are and that's what's important most universities are now doing data ethics and AI ethics there are so many Frameworks that are out there the challenges there's not that commitment we see that you know big Tech they have actually unveiled and deployed some of the prettiest and most glamorous ethical and AI Frameworks whether or not they adhere to it probably not we've seen the ethical uh window dressing and the the ethics washing and just you know decorating those those windows with ethics while doing things that are unethical but just saying this is my data and I need to know what's happening with it you may never own it you may never be able to monetize it but you can surely make a decision the way it's being used collectively to build our future great presentation thank you so we just went through an election and you mentioned the AI Bill of Rights which I have interviewed I don't know exactly what's in it so um you also mentioned the fact that you come from a criminology background so is there an accountability requirement or should there be one for candidates for any office to say where their position is on what tools would they use how much AI would they use would not use is that part of the AI Bill of Rights or you know it's not but it's an excellent suggestion and a great recommendation that we can make because it's important it's important but what we have realized is many of our elected officials really don't understand data science and algorithms and the ways in which Technologies are being deployed the good thing is that a few weeks ago if not a few months ago Stanford University created a program designed specifically for legit legislators persons in Congress the house to really upskill them in real time because they have to make the decisions but the challenges technology is always way ahead of our legislators so when they sit in the room with the technologists and the the CEOs of these big tech companies and database data scientists they're not engaging at the same level so it really is about that upskilling in real time so it's a fantastic recommendation and I think it's one that could definitely take us in the right direction thank you yeah hi thank you for your presentation I really enjoyed it and thank you I'm a data science student and I wanted to ask if um so we we found out that the the AI and the Technologies are flawed right so I was wondering like um are there still places where this these systems are still being used to make decisions and how how are you how how is the bridge being oh is the gap being bridged between people that uh say you said the politicians the the people in the Academia for data science and you know the people in the second software engineering industry or Tech so how is how is it that how are we Bridging the Gap between oh so we have a flood um technology here are we still using it or rather how are we making sure that the data scientists that are coming out or we learned about data ethics in in a very small segment of class in a in a profession like in an ethics class but we don't I I don't see like if other universities have Incorporated this in their say in software development software developers especially they don't care about ethics so much so I was wondering how is this uh Gap being bridged with all those stakeholders and also how others flawed Technologies still being used and how the people that have been affected how has that been mitigated thank you so I will say that our data sets are flawed because we're flawed and our data session representation of who we are in our society so what we have seen would be those inequities and inequalities and systemic challenges baked into our data sets we have seen some communities and organizations and data scientists coming up with technical tools to de-bias the data but we're seeing that's not enough because we've got to do buy us our thinking most of the time as technologists and designers as we build these Technologies what we're also seeing is this requirement for ethical data scientists and we're seeing that becoming part of compliance at many organizations they want ethical data scientists because what they're also realizing is yes we have got to protect those rights and we've got to ensure we minimize those risks but I think what many organizations are realizing is the cost of those risks you know a media attack an expose Regulators are now breeding down your back of course Financial damage all of these are ethical risks so most universities if not all universities who are doing data science are putting that element in there I mean I teach Big Data ethics and AI ethics and and bringing that kind of knowledge to our data scientists because when you think about diversity and when you think about inclusion and Justice and understanding those traumatic memories and just understanding just drilling deeper into the data sets what you are doing is stretching your own imagination as a data scientist and it means that you are now bringing a more critical uh more critical thinking to the creative process so it makes you a sharper data scientist and it also makes you less of a liability for an organization once you start to work within that compliance framework so it's being demanded it's being demanded and you find many data scientists who have graduated with master's degrees and even phds are now thinking about am I doing this ethically is there something else I need to do another certification to ensure that I understand questions around accountability and questions around transparency and the fact that if we are building these algorithms and we're using and designing these black boxes that question of opacity of the Black Box undermining full transparency so there are so many questions you know uh trade secret proprietary rights the things that we are thinking about from all different uh you know areas and and one of the things that we've also got to encourage is that interdisciplinary imagination because the more and more disciplines we have intersecting with data science and AI it means that we have an opportunity to build something that's truly inclusive at the same time so thank you for that question thank you for the presentation it was very interesting I have a couple of questions about data health so one way that you talk about data health is that the biases are informing that the data existing data sets that need to be cleaned up um so the first one first question is so if you're going to clean it up isn't that a human action to clean up the data set and wouldn't that have human biases you know like I don't I I'm trying to think of like how what's the the most democratic way to clean up the data set so that it removes as many biases as possible and then a second related question is and I can't remember the the the phrase that she used but I listened to a talk by Amy Webb from the future today Institute that talked about digital IDs where it was you know to to create digital thumb prints for each individual so that that data set because right now our data set is you know Amazon has one eBay has one Spotify has one Netflix has one and they're not connected and I think what this woman was saying is that in the future there will be a digital ID that can be really you know associated with the ones human being and I wanted to know from your point of view in in data Justice would that be a good thing or would that be even easier to weaponize and harm people thank you for that fantastic oh those fantastic questions so I will say this the one thing we know about AI is that we don't know about Ai and that's the challenge so we can come up with all different types of approaches and solutions but until they're deployed do we truly understand the kind of impact it will have so anything could sound like a good idea until that good idea comes or intersects with the community with society and that's why we need to always think about those long-term impacts when it comes to the data sets uh you know bias in data science is sometimes good and sometimes challenging that's that's what bias is about but what we need is that critical data science perspective that brings the requisite level of Consciousness to know that we have got to be ethically Vigilant to know that we've got to be diligent in the ways in which we are using data science so yes we can do the human in the loop and the human in the loop is great but sometimes the loop is too big for the human and sometimes the tech companies are creating loops and running those groups all around us so it really is about that knowledge and that understanding and really also includes just an approach to the ways in which we are doing data that allows us to understand the limitations as well so we may need more disclaimers there are some data scientists and technologists and philosophers who say it's time for maybe a Hippocratic oat to the ways in which we're doing data science or as we have nutritional labels we need to have Digger labels that outlines how we are using the data or uh just different ways in which we understand the information that's being used the challenges that these are all great questions and because it is still so new it is important for all of us to be in the discussion because we don't have the answers but certainly by creating a space for have us to have these conversations and to have these kinds of conversations it means that whatever the solution is we come up with it's definitely going to be one that's inclusive and diverse so uh fantastic questions I will not have all the answers because it is so new and it's happening in real time but I'm certainly a part of the conversation and I think that's what we all need to do just get involved in that conversation because not until it's deployed do we really understand the impacts of of AI and algorithms because we still don't understand many computer scientists are yet to truly understand or able to explain why these algorithms are making the kinds of decisions that they are making and an easy way not to explain it is to say intellectual property rights Trade Secrets proprietary rights because we still do not understand what's happening in that black box and it is the opacity of that black box that really undermines true explainability and true accountability and true transparency so there's so many challenges but it's an amazing technology it's a technology that's transformative it's a technology that can help us build an extraordinary future but we've just got to use it wisely and cautiously and of course ethically so thank you thank you for your conversation tonight so I'm curious in lieu of a better system that hasn't happened yet um Banks and Loan Companies when somebody's going to buy a house I know that one of a big problem and we had this here in Michigan with a local bank are discriminations against people of different genders and of different races would it just be easier for banks and Loan Companies to Simply omit those data points and just remove the box to check as to whether or not somebody can get a loan or the size of a loan simply based on their race or their gender would that just be better just to remove that question altogether or until we can come up with a better system of calculation or should it stay the same and we should just stay the course and try to make something better in the long run thank you fantastic question I would say give everyone a house but I'm not a banker right everyone deserves a house you know these are the questions that we're trying to deal with so when when you know I work a lot in the c-suite and I have lots of conversations and uh with individuals in finance and credit and these are some of the conversations they're having you know how do we do this you know we want to use these algorithms to personalize our services but as we personalize we realize we're excluding and this is why data science needs all of us in that conversation because at the moment I'm sorry at the moment they've not come up with the solution but these are the questions that we need to ask and these are the questions we need to have because banks are part of the community they are stakeholders they are not Lords onto themselves and they need to address the concern so they need to take a deeper look a more intimate look at the data in their communities the communities that are supporting them and understand that they need to provide services that are Equitable so it's really conversations that need to have so maybe uh the next conversation would be bringing all the banks together the credit unions the financial institutions and find out how are you doing data ethically definitely given the extraordinary amount of data that's being collected and that's why I said it's so challenging exactly because of those proxies and that's why we need to have those conversations because it means that that data is not serving the community if people are going in there to get a loan to get a house it's about those legacies and if you want to build up your community you've got to understand your data in obviously different ways so you need to have a more creative approach to the ways in which you're using data so financial institutions have got to be accountable to the communities that they serve first question is I recently talked to a user experienced person from uh I think meta and uh he was telling us how AI is profiling people without them having to enter any information so I just have my first question is what's the safest way to not be profiled by AI there is no safe way to not be profiled by AI that's the challenge with this technology there is no safe way we are being profiled once we are using technology we are being profiled those data points are being collected and profiles are being designed and those profiles are being used to make particular decisions uh the minute that I started to do data science and data activism and work as an AI ethicist I lost all my Facebook friends they all disappeared from my timeline and only neuroscience and Ai and that's it they all disappeared I just lost total contact with all the things that was happening in my small community so you really don't have a way the best way is that you can protect yourself and you could understand how you're being profiled and you could understand the implications of that and you can understand your rights when it comes to data and you could understand that you need to be aware and you could also educate persons in your community so there's no way not to be profiled but you can certainly be more diligent and you could be more situationally aware of how you are being profiled okay uh my second question is I might have missed it but is there an exact number of how many times AI has falsely incriminated people and what is being done with these numbers well there's no exact number because most people who've been the victim of an algorithm uh probably don't know there are several individuals who realized in the criminal justice system that they were being denied parole and a few of them decided to take those vendors to court in court the question became proprietary rights and Trade Secrets meaning we can't tell you what's in our algorithm versus human rights and what we are saying is if we're deploying these Technologies and these risk assessment tools in the criminal justice system where it's about life and Liberty it could also be about death we have got to bring a very ethical perspective we've also got to educate criminal justice practitioners we've also got to educate individuals who are procuring technology because many of the individuals who are procuring these Technologies in the criminal justice system the educational system in many of our systems still don't understand the kinds of implications these Technologies can deployed and the kinds of consequences so it's really a broad-based approach that really brings you know awareness and education and literacy to the ways in which we're doing it so at so many different levels we're finding challenges but there is no full number and I think we are kind of afraid to find out what that number is because those are lawsuits I think many people are not expecting and would not want all right my final question is you said educate people about AI right AI is fairly new so how long do you think it's going to take to educate enough people to make a real difference well it's going to take a while because the technology is changing as it's uh being developed as well but we've got to start the process because we now communicate in data we now communicate with algorithms and if we're using this as a form of communication we need the requisite level of literacy so data literacy AI literacy Futures literacy is also critical because what we're using technology to do with design Futures and we have got to understand how this technology how data how AI not only changing the ways in which we communicate but also deciding who communicates with who and who's going to benefit from a particular type of communication or who's not going to benefit so it really is very critical for us to start to have those conversations and for us to start to educate from you know kindergarten go right up even speaking about AI ethics uh because this is the language this is Who We Are I have a question um how to overcome a political oppression specifically in organization have to overcome um kind of this culture of silence culture of ignorance culture of conservatism when you try to a culture of disengagement how to overcome those kind of challenges when you work in encourage them in organization sorry and for example try to deliver evidence-based data and you don't have this kind of uh your voice heard an upper level Administration when there is a hierarchy between lower and upper level and you kind of in a level of administration and you try to um to speak and um there is a a very rough dialogue how to overcome this kind of uh when you when you speak about delivering the data and Justice I just personally have this kind of challenge to over have to overcome those challenges [Music] in in internal organizational level thank you thank you so much and an excellent question and what we hope is that organizations will develop an ethical culture when it comes to data and algorithms and Ai and some organizations are doing it and some organizations a challenge now depending on where you are in that organization you may have more of a voice to really uh you know voice these types of concerns now I'm not going to tell you to just go in there and go into the board charge into the boardroom and tell them they're doing it unethically because you may not want to take that chance but if you get the opportunity to be part of a design team uh one you can bring that knowledge that you have about ethical Ai and ethical data science into the space or even at the level of a new employee uh when we have those conversations with HR when we have those conversations with our managers you can say that that's something that you want for professional development why don't we uh you know include an ethical approach to the way ways in which we're doing data science or the ways in which we're doing evidence-based research or designing evidence-based strategies uh saying that that could assist in in making us you know uh more effective and more efficient uh professionals so you can ask for it for part of professional development or you can just uh you know ask the organization uh you know to expose uh your team to that as I said I would not um you know advise anyone to just go into the company in the morning and say hey we're doing this unethically we need to do it differently of course there's whistleblower protection I'm not going to tell you to blow the whistle either but what I'm saying to you is that you know they're trying to develop whistleblower protection in Tech and legislate that as well to ensure the tech workers are have a voice there are some people who've decided and who've done it uh you know and who've done it uh you know in a very uh sort of uh you know Brave way very heroic way but the main thing is is that organizations are waking up to the fact that we need to do technology ethically and they're waking up to the fact that we need to bring all employees into that conversation not only the data scientists and the designers and the uh the system thinkers but everyone you know from the c-suite to Main Street that conversation needs to happen so thank you for that question so um I wanted to kind of drill down a little bit in the the example you gave on the parole and as a statistician we would kind of cringe at a single number and not having a level a type of uncertainty measurement that goes with it I'm sure that you know the language I'm speaking and I also wonder sort of with AI you know you're taking large sets of data and you're making a prediction and usually those predictions are might be fairly accurate for a large group that has those characteristics but then you're applying them to an individual and I think we probably all know that individuals vary a lot more than group averages vary so I wondered is there in the discussion of algorithms and algorithmic rights is that discussion part of it sort of the there's uncertainty with these algorithms and we need that measurement of uncertainty as well fantastic question and of course I'll say this the conversations are happening they happen in particular in the school of data science at the University of Virginia because my dean of academic Affairs and faculty is a statistician a Jeffrey blube and it's a conversation he likes us to have all the time the challenge is this you know it's really about bringing an ethical perspective it's understanding how we do the computation how we do the science how we do the math it's also understanding that uh this is uh still so new and still so opaque but really ensuring that our statisticians who are assisting Us in the ways in which we design and deploy these algorithms also understand Concepts such as a data trauma and why we need a Justice oriented approach and social justice oftentimes conversations that statisticians don't like to have so it's all of us having these uncomfortable conversations to ensure that we do this technology right so the social justice experts and the statisticians if they come together that table I do believe we will have the kind of inclusive ethical and responsible trustworthy technology that we deserve what are some pitfalls like we all know any reasonable person knows that a a really bad data set like you know for example the redlining how the red line housing Maps that's obvious example no reasonable person would say that that was good data but what are most the time bad things get done by good people with good intentions what are some pitfalls that you see in in big data sets that are seemingly innocuous that but cause trouble I'm guessing you see some pretty interesting examples of good ideas gone horribly awry that uh how How might people see that out see that kind of thing coming if you've got any insight on that well I don't have as much insight as you are asking me now but I will say this to you it's again uh really about uh the ethical vigilance and being diligent bringing that you know really sophisticated level of due diligence and it's about your thinking and it's one of the reasons that I encourage this you know interdisciplinary imagination because the more spaces and places that we've been the more of an understanding we bring to the space so oftentimes if I'm working with other data scientists and we're thinking about ways in which we can do data science better I'm bringing that knowledge one of the things that I always say in particular when it comes to Ai and algorithms they don't have lived experience they don't have emotional intelligence they require us to be a part of that process for them to make the kinds of decisions and I always say that we don't ever want to get to that place where we will need an algorithm to teach us what it means to be human again so what we need to do is bring that understanding that thinking that consciousness uh to design space to the deployment space to the ways in which we're adopting because we're never going to get the perfect data set we are never going to get that but what we can get would be individuals who are designing responsibly who understand you know the kind of knowledge additional knowledge supportive knowledge that's required but also understanding the limitations of the technology and why we need those disclaimers why we need to educate people on what this technology can do and what it cannot do what this data set is represented representative of and what is not representative of so it's really as I would say there is no perfect answer no perfect data set there is never going to be a perfect algorithm well there could be is the fact that we've had all the conversation we need to have around this and we're going to try to build this thing as inclusive and ethical as we can and if we falter we're going to take responsibility for that I was wondering if you could give us an example from your extensive work in in communities um of just uh some one or some entity who is actually doing something stopping some practice right to to disrupt right this data and Justice as you point out is it can you think of a of a of a concrete example just to to Sure fantastic and there's actually a documentary out coded bias that features uh Dr Joy bulum Winnie and uh of course uh Kathy O'Neill who rode weapons of math destruction and Dr safia Noble is also in it and she wrote uh algorithms of Oppression as well as Meredith Broussard who wrote the book artificial intelligence and they're all in that fantastic documentary and of course there are these tenants who came together as activists living in an apartment building in Brooklyn New York who decided that they were not going to let their landlords deploy facial recognition against them and they came together as a group as social activists they worked together with Dr joy bulumwini and other data activists and Ai ethicists and data scientists and they dealt with that situation in court and they realized that as a community we were not going to accept this because this technology was discriminating was criminalizing and victimizing us and as residents living in this apartment building we were not going to accept that so that's one Community there are other communities our indigenous communities that are coming up with so many different ways to ensure that we get data governance right and we get data stewardship right and we get data protection right and I think all over there are groups that are coming together in Detroit in Minneapolis there are so many groups that have come together activists and individuals working in the local governments base to ban things like facial recognition and more and more we're seeing those Coalition coalitions and coming together in ways in which we can get this thing right because that's what we've got to do so we understand the extraordinary benefits and I am all for the benefits I'm all for using technology I'm also designing approaches to criminal justice using virtual reality to bring more equity in that space But while I'm committed to being part of the design process I just want to see those Equitable reliable you know resilient and sustainable legacies being created at the same time many many things thank you so much for your time [Applause]