The Ethics of Persuasive Tech & Hypernudge | PAIR UX Symposium 2018

[00:00:00]can it excellent can it is the designer and the North Lehrer and he's been looking particularly at future ethics he's been advising different companies like Samsung Twitter around the ethical implications of Technology thank you thank you all design is interested in behavior for interaction design that really points in two directions that interest one we're interested in designing the behavior of a system responding to user input but we're also interested in the medium of human behavior and for all design what we consider good design is often design that directs behavior appropriately a design is successful if the user can react in the way that we'd planned for that we'd anticipated a good interface for instance is one that highlights the right information in a system so that the user is steered to the next step in the process if we say enlarge button on a UI yes it makes it more visible but the main point is more people will click that because it's about behavior then that means design is laden with ethical impact indeed you could say that design is applied ethics I think it might have been Cameron Tonkin wise who argued that design is morality practiced with the hands now sometimes that relationship is explicit and obvious if you're designing razor-wire for instance you're saying that people have a right to privacy and a property such that actually people should be injured for trying to contravene it if you design speed bumps then you're saying that a car cannot travel down this road beyond a certain speed without damaging the the owner suspension there in these instances we're delegating moral control to the artifacts in the environment but every act of design whatever the medium is about specifying the future it's about making a claim for how we think the future should be when we choose one preferred future we also discard thousands of alternatives and if design is really around deciding what to make and if ethics is really about deciding how to live then there's clearly some kind of relationship between those two things so designers try persistently to persuade people to act in desirable ways and this is not a rational art if it were it would be fairly simple we'd present the options available and we'd leave it to the user to make the rational choice but we know well that other factors influence decision-making emotions and defaults framing so designers have to appeal in part to the unpredictable aspects of human nature in essence to human weakness hence nudge theory which is all about changing behavior without changing the options available it's all about the power of framing and positioning so we know the common examples it can change defaults to encourage opt in for say organ donation you can move the healthiest snacks by the tills to encourage people to make healthier choices a nudge of course has been terrifically popular in the public sector but Silicon Valley I think likes it as well nudge doesn't trespass on the individual freedoms that the industry holds dear but it's still a very potent way to liberate people from their time and money nudge generally I think has a positive reputation because its applications are seen as positive we always talk about it in relation to things like weight loss or smoking cessation financial prudence but of course any persuasive technique has a potentially less healthy purpose designers I think are particularly familiar with this dichotomy our industry designs appealing cigarette packets and the calming hospital signage for the smokers last days as currently there's panic about persuasive technology becoming addictive it's not one I'm quite so concerned with I think the lines between a product that's simply so good that we want to come back to it so well-designed and one that's explicitly designed to capture attention I think those lines are blurry there's also not so much direct evidence at the moment that technological addiction is a separate disorder from existing ones however it's clear that chasing engagements that essentially trying to persuade people to reuse our products on a habitual basis has led to some very trashy users of Technology this of course is the fundamental goal if you like of tristan Harris's time well-spent movement to interrogate that potential downside more interesting to me though is the politics of nudging because we find there's a 2015 study that attitude to nudging actually vary quite a bit across the political spectrum libertarians particularly are skeptical of nudge because they they regard it as the top of a slippery slope but we find that whatever someone's political leanings the biggest way to change their mind to change their appetite for nudging is to show them examples that they politically approve of or to say that the people trying to institute these nudge policies are politicians that they happen to support the same study also found that nudge can induce antagonism some people mostly right-leaning people deliberately chose the alternative option when they were told that they were being nudged essentially a protest vote another study found that the source of nudging matters US and Israeli participants in this study generally disliked nudges that came from governments and authorities instead they preferred employers to be doing the nudge in in Germany there was no such difference and we rarely talk about the politics of these persuasive angles even though Thaler and Sunstein the architects of nudge theory went too far to call nudge libertarian paternalism technologies are of course great platforms for designing behavior change they sit atop oceans of data online behavior sure but also an increasingly data fide world typify by computer vision by audio capture IOT in this the smart city has substrate for urban sensing we all know they know banal cliche that data is the new oil but that contains some interesting metaphors we now talk about oil with these liquid figures of speech it's a deluge it's a torrent it's an uncontrollable natural source it's no accident of course that oil is dangerous to store at high volumes and that it has a habit of leaking we also now have the Kombi a tional ability to make sense of this ocean of data we have machine learning and analytics tools to make sense and to act upon these systems and this offers a significant scale it means we can abstract persuasions of the technologies all around us giving us higher impact at lower cost but the common examples of technology and it's persuasive potential they're mostly Zhijun and harmless think of you know wearable fitness trackers apps that remind you to drink more water and so on but we're starting there to find deeper and more eye-opening prospects carrying young who's their legal scholar based in the UK says that we're entering an era of what she calls hyper nudge this is a persuasive system that's networked so that it can learn from millions of people it's dynamic so it's able to shift strategies unlike for instance mass advertising and it's personalized potentially down to the individual so this is a system that knows intimately you and your data it can adapt it to your behavior it can change its mind if an approach doesn't work and it will therefore know exactly which buttons to push this could be an irresistible manipulator we already have Croods variants of this already quite effective and quite scalable another study found 50% uplift in facebook ad click-through simply by tailoring the copy in the image in the in the ad for very crude personality types interpret interpolated from a user's likes this is a fairly manual process but it's clear how this can be automated 50% is obviously a very large uplift through this particularly crude human operated algorithm Amazon already send millions of automated nudges their affiliates and this suggests of course a significant ethical question at what point does a nudge become a shove at what points does this act of persuasion become unethical and that's particularly tricky to pin down the ethics of persuasion are complex you always have multiple actors for a study you have at least to a persuader and a persuade II Daniel better Chesky at Noranda early theorists a persuasive technology put it very nicely persuaders have always stood on uneasy ethical ground if a serpent persuades you to eat a fruit does culpability fall on you or upon the serpent technologies dilute this further we still have the persuader and the persuade D but now technologies the medium for that persuasion to take place and Tec itself has many layers morality seeps into the whole technology stack so if I have an AI that per Sade persuades me to commit fraud who's responsible well me sure I should bear some liability but perhaps also the software engineer or the platform company that allow the app to exist unchecked in their App Store or on their operating system maybe at some point will even blame the AI itself this requires us of course to admit what's been hinted at a couple of times a day that technology is not neutral we don't just build mere instruments to fulfill tasks back in 1980 London winner philosopher of technology wrote a famous essay now entitled to artifacts have politics his conclusion was essentially well damn right they do and Robert Moses was the winner gives the example of Robert Moses rather as the New York City time planner back in maybe the forty something like that and Moses was a racist and although it's slightly disputed it is alleged that Moses built intentionally low bridges covering the freeways on the way down to Long Island beaches so that black people who traveled mostly by bus wouldn't be able to get there even a bridge this hulking monstrosity of hundreds of tons of concrete and steel and iron have moral impact because it's a mistake to separate technological capabilities from human capabilities these things acted together we're into woven hybridized actors things change what people can do and how they do it it is true that guns on their own don't kill people but a gun burn certainly can so we have to reject this idea of neutrality we have to accept the inevitable social political and ethical outcomes of a clear ethical risk in all this of course is that we eliminate free will that's a hyper effective persuasion diminishes human agency which is something that lies at the heart of almost every popular dystopia and the power dynamics of hyper nudge strike me as particularly worrying because they have the advantage of scale they can learn from millions of people because they can shift strategies at will - large algorithms are highly potent but essentially invisible diluted by the technology stack using obfuscated data flows and operating on the thresholds of consent and this creates a significant power imbalance if we aren't careful we'll build systems in a way that we won't know who's persuading us we won't know their intent and we won't know their methods this means that people will have no recourse to collective eyes and to contest these systems because they won't even know that they're operating we will become blind to our own manipulation which of course makes this a terrifically valuable prospect for the high stakes games of politics we all know far too well how this is heading the recent headlines about Cambridge analytic a clearly point in that direction the nature and structure of connected technology and particularly the web already privileges certain belief systems hypertext has a way of breaking down centralized linear narratives it encourages instead apophenia the habit of imposing relationships on unconnected things so you could argue that the structure of the web already affords conspiracy where this gets particularly alarming for me is if we combine that tendency with evidence collapse with falsifiable audio and video so we can have soon an AI that knows how to manipulate us individually and that can generate the evidence required on demand to support that manipulation it's a particularly scary prospect now we might counter we can say we can build trust based technologies things like cryptography and blockchain technologies and you know maybe that will help alleviate that situation but the complex technologies we may end up very likely in a situation where they're available to only the rich or the highly digital literate and information Anarchy still reigns for the rest as always what the super intelligence and powerful do with AI is a bigger threat than powerful super intelligent AI so what can we do about this it's all very well to acknowledge these issues of course but I think we have a bit of a moral duty to try and mitigate them and I'll suggest just four steps here first and again I think this is something that's been hinted at throughout the day we need to materialize the invisible so many of Tech's ethical dangers stem from the invisibility of these spaces be that intentional or otherwise by shifting persuasive acts into the visible spectrum people can make more informed choices about what's happening and this is already happening in other areas GD P R is gonna force a lot of data flows to surface the explainable AI movement will illuminate the reasoning for algorithmic decisions people may argue that this might make persuasion less effective frankly I don't care I think the power of these systems is potentially too great for us to be too concerned by that and know what I'm talking about materializing this stuff I don't necessarily mean increasing the complexity the complexity of our systems I'm talking about making information available rather than mandatory creating extra screens or interfaces or affordances that people can interrogate if they want to know more there's also a host of ethical theory we can bring to bear I've only got time really for one here this has come this comes from John Rawls in his book a theory of justice rules talks about the veil of ignorance the idea that we should design a system as if we don't know our eventual place within that system so if I'm designing say a welfare system I should design it such that no matter where I end up whether I'm an administrator of that scheme whether I'm a taxpayer whether I'm a welfare recipient I should look at that system and think I've got a fair end of that deal I think designers generally get this this is a really a forcing function on maybe not empathy that misused term but at least getting out of one's own head and seeing a problem from multiple points of view I also think we should make a distinction between an informational and a structural nudge this this idea comes from luciano floridi and he distinguishes the two in this way he says that a structural nudge is one that changes the nature of the action available to you so if I run a cafe and I want people to eat more healthily then if I change the order in which food is presented or move things to the back of the store or move them high up and so on that's a structural change now the downside with the structural change is it can lead to almost forced compliance if I put the snacks right on the top shelf well anyone below six-foot is going to struggle to reach them by contrast and the informational nudge only changes the nature of the information on offer not the action itself so the example here will be changing the labeling not moving it to a high shelf that generally poses less of a threat to free will the user is better able to retain their freedoms and this aligns generally with our social hunches about nudging in 2013 yes another study people found that they were more favorable to nudges if they used conscious process so things like signage than unconscious nudges like room layouts again this relates to visibility it was easier in those situations for a user to see how the information environment around them had been shaped finally I think technologists have a role to create counter technologies that can test a protest and the hamper overreach where it occurs and this is really analogous to the rise in ad blockers maybe we will see persuasion blockers browser warnings for this kind of thing maybe even some some sort of nudge traceroute to expose who's trying to get you to do what within a system and of course there is a potential role here for regulation but that will only happen with continued pressure from the industry from the public and the press alike to see this kind of real change it may take internal pressure and occasional whistleblowing and campaigning I will say that if you feel comfortable safe and respected in your industry in your work you're in the perfect position to use up some of that goodwill for ethical change we have to try and continue this push for change in the industry it's it's exciting to see over the last 12 monthly months the increased focus and focus on our ethical implication but we need to keep that going I think we're going to need all the help that we can get I have a book will be out in September your L is there and I'll be talking about it a lot on Twitter and thanks for your time we've got time for one or two quick endeth earlier we would skip Q&A for the panel but I'm not gonna I ran short exactly my ethical responsibilities so what are two questions for Kenda thank you I work in public policy and I want to really congratulate you on this work that you're doing because it's very important I just want to say if I look at the evolution of digital technologies it went really from a great promise to today in some areas an existential threat and what I see here in the room are incredibly well-intentioned people so my question is how could it all go so wrong and how can you build in the community more of a sense of sort of strategic foresight in anticipating some of these challenges better that's a big question what went wrong a bit of hubris a significant lack of diversity I'm very very strongly believe that diversity is an early warning system for ethical harm I recognize it's a highly politicized issue at the moment but I make no apologies for bringing it I think a team that's homogeneous is always going to overstate the value of their work for people like them and to understate the value of the potential harms of their work for people not like them what was the other part of the question sorry oh yes how come yeah essentially how can we try and address this right what are some techniques I'm turning a lots of speculative design on this actually um there are tools to if not predict the future but to open the possibility space of potential futures if you like and it's sort of highfalutin language I'm fairly convinced now that speculative and critical design is probably the next fields that interaction designers are going to seize upon and partly ruin but I think there's something very interesting in those techniques and I'm a lot of the work I'm doing with clients now is to use those structure techniques to just scan the horizon bit better and to better understand unintended consequences and once they're exposed then of course you can start to try to mitigate them so yeah they're not magic answers codes of ethics or a whole other thing may Paul rant about those a bit later but yeah you know we aren't the first people on these shores academics people in STS people in the philosophy of technology have been looking at this for decades and it's about time we started listening to them as a community shall we shall we analyze yes penalize thank you next we're gonna have a panel around the opportunities and challenges of UX and I and Matt