Philosophy of Gamification: How Social Media Shapes Values

[00:00:00]so I've been thinking a lot about echo-chamber is an online outrage so here is the big idea for this talk my interest is in the possibility that we're simplifying our morality and our values for pleasure they're having more complex and nuanced values is a little more painful to go about in the world and having a simpler one is a little more safe secure communal and I'm even more worried that because of this possibility one of the things that might happen is that people out there are consciously designing systems to encourage us to simplify our value system ok the idea is this is a kind of conceptual engineering idea I'm worried the big reveal that I'm supposed to say for the end but I'm gonna say now the big worry I have is that people are engineering systems designed to create sticky narrow-mindedness right that's the idea that's where all the stuff is headed so first echo chambers probably the reason I'm here and talking to you is because I wrote this thing that went by apparently philosophy pieces can go viral this thing went viral and in more readers than I could even imagine and this article was mine was based on this book which I think everyone should read by Kathleen Hall Jamieson and Jessica Pella on echo chambers which is a study of Rush Limbaugh on Fox News in the 90s and early aughts and how they operated so one of the things I noticed was everyone in sociology and political philosophy and journalism were using these two terms as synonymous one is filter bubbles and the other is echo chambers and if you go back and read the original text that introduced these ideas I'm a philosopher so I'm always doing this like this this is the book where the idea of echo chambers came from you'll discover that they're actually totally different concepts okay so a filter bubble which an author named Eli Pariser made this term famous is something where you don't hear the other side okay the idea of a filter bubble is all your friends on Facebook and Twitter have your political alignment so you're never exposed the other side right it's a not hearing in Jameson Capella's book they talked about an echo chamber or something where you were taught to distrust the other side right and those are two totally different concepts right and I think it's very clear from an analysis of what happens with Fox News and Rush Limbaugh that people who are inside the Rush Limbaugh world actually are perfectly have access to and hear counter-argument C on the other side they have they have a story that tells them that the other side is corrupt on trustworthy there's a liberal elite in charge of the newspapers and there it's all made-up fiction right so and an echo chamber in the original text and in my excavation of it is a structure or insider are taught to distrust outsiders and where everyone who doesn't share your core beliefs is an outsider right this has the structure this should be a familiar structure this kind of has the structure of a cult right this is what cults do they teach people on the inside that everyone on the outside is malignant and out to get them so it's an isolation not of like avenues of communication but it's an isolation of who you trust so one of the most interesting things about echo chambers that comes up in the Jameson capella is that echo chambers are almost always associated with conspiracy theories so there's a good explanation of this from a philosopher named Andre Begbie here's this great paper that I found really useful in understanding this called evidential preemption and this is how it works so an evidential preemption let's say I'm wrestling both right let's say for the moment pretend that you're my followers and imagine I'm Marcia Limbaugh and I say see you look everyone out there in the media is out to get me so I make a prediction when you go out there and when you read other people talking about me they're gonna say I'm a liar they're gonna say I'm getting all this wrong but that just a conspiracy out to get me your hero right so notice what I've done I Rush Limbaugh have made a prediction about what other people are going to say and then you go out there and then you see that CNN says those same things about me and my prediction is confirmed so you have a further reason to trust me that makes sense so it's kind of I mean I think of just like intellectual judo not only am i like blocking incoming evidence but I'm making you trust me more in that block right so this is gonna be really important to just lodge this in your head so here's the question I had after I finish writing this piece we want to talk about today why our echo chambers and extremely vicious political polarization flourishing in today's communication environment this is a polite way to say Facebook and Twitter right okay what's the relationship between new media communication technologies and echo chambers so I left I have no idea it's a serious question for me because as you can tell echo chambers they are cult mechanisms we've seen them they've been with humanity as long as we know right why is this this explosion now it seems to be happening online so what's the connection and in particular what is this if you read if you well I was researching this I spent a lot of time reading American alt-right websites like Breitbart and I read the comments section every day for three months and I actually wanted to die I hate to stop for like put like emotional and health reasons right but you find this too and the students conspiracy theories and moral outrage right in this particular note I wanted to understand by the way I should also say I also think there are echo chambers on the left and when you go into those you find exactly that same mix I'm not saying by the way the two are the same as far as I can tell the is this enormous gargantuan mammoth but they're there they exist on both sides and they hate seem to have that same to do in that same flavor that's why I want to understand so here's our method is a method that people say comes from a philosopher Dan Gilbert Ryle but you actually find it in Schopenhauer it's called creature construction we're gonna imagine ourselves into the role of a large scale belief manipulator okay imagine you want to take control of people's beliefs right by injecting ideas into the discussion how would you do it so put on your most evil hats and we're gonna look for techniques of belief manipulation so they're gonna be three parts of this here are the three different streams trust me they're all gonna sound different and we're gonna get to the same place hopefully the first is false clarity the second is porn I told you we're gonna talk about porn and if so it's why you're here right we're gonna talk about porn in here philosopher talk about porn I wrote a paper about porn I wrote two papers about porn sorry in games this is by the way this is apparently my brand now I've written papers about games and papers about porn so this is what I do now gamification of Twitter and hopefully these will all connect for you so here's the first idea about clarity and false clarity this idea I've been having the clarity sometimes is a thought ender so if you look at the history of philosophy in the history of a lot of what it is to be rational you get these descriptions of these like endless processes where you have to figure out everything and get to the bottom of everything but we are not beings that can think infinitely we are limited beings right we have a small amount of time and so we need to manage our cognitive resources we know we need to know where to spend our intellectual efforts or intellectual focus right so we need a quick and dirty way of knowing when to end an investigation without having explored it all the way to the bottom right if every line of thinking you had explored to the bottom then you would never get anywhere you'd have to it you would be like a philosopher which we don't actually do things successfully right most people in ordinary life need to know quickly when an investigation is probably thought through enough so we need be heuristic if you don't have this term this is a rule of thumb right we need a quick and easy quick and dirty principle that lets us figure out most things not perfectly but quickly so here's my proposal we use a sense of confusion as a heuristic for continuing investigations and we use our sense of clarity as our heuristic for ending investigations that make sense so when things feel clear to us we stop thinking about them so if that's right there's a vulnerability here and you can start if you put on your evil epistemic manipulator hats you can start to see what that vulnerability is so if you're evil this is what you should think if you can give a system of thought a feeling of clarity people will stop thinking it about it they'll stop investigating it right so you want it to feel clear this is kind of a manipulation of attention so this top thought terminate heuristic I think of something like an invisibility cloak for your brain right so here's a way to put it john lecarre a when I give this lecture in America to American students no one knows who that is I'm assuming in here most of you know john lecarre a is spy writer also an actually worked in british intelligence so he knew what was up he said in his famous series and George Smiley every book you would remind you the best spy didn't look like James Bond the best buy was boring normal bland something that your eyes glanced over right something that said it was uninteresting nothing is worth thinking about here similarly stage magicians if you know how they work the way stage magician works is if the stage musician wants to do something down here they make this have look really interesting and flashy and they make this hand seem really boring and dull right they know how to send signals of boringness and dullness so the idea here is this make sense you if you control these signals of interesting boring right confusing and think about clear and settled you can start to manipulate people's attention so here's a test run a lot of this is philosophy but there's evidence from cognitive science that this actually already happens so there's this charm from cognitive science called processing fluency processing fluency is just how hard or easy it is for something you think about something right it's easy to think about it's processing fluent it was hard to think about it's not so we use processing fluency as a shortcut to acceptance so psychologists have found that the easier task is to if you look at a purpose like a proposed belief the easier it is to understand the sentence the more likely you are to accept it and the harder it is to understand the less likely you are to accept it there's actually a reason this is a good rule of thumb this is a good heuristic because most of the time things are easier to think about if you have experience with them and things are harder to think about if you don't have experience with them right so it's a good starting rule of thumb but it can totally be manipulated for example clear replicated results show that when something is a idea is first of all if you see an idea write a sentence you've never seen before and you're asked to rate its reliability how likely it is to be true you'll all assign it something and if we show you the same thing a month later your belief and it will go up because it's familiar right it's easier to process here's another thing this is the thing I find most terrifying almost everything is more likely to accept a sentence written in a legible font then a hard to read font it is it's easier to process right seems so it grabs this heuristic at bars this can be gamed right evil manipulators can take control of lists this is what slogans are this is what motto is are this is what in turn even net memes are right they work because it's presented in a familiar format that familiar format gives you a reason to accept it so next step that's that's the first little warm-up so let's be hostel belief manipulators we want to create a system that maximizes the feeling I'm sorry this is slightly out of order because I'm dumb I just had an incredible idea this morning and try to rewrite part of my talk so this is not something you're supposed to do so to understand this - so what I want a hack if I'm an evil manipulator and I want to give you a sense of clarity I want a hack your sense of understanding things right I want to make it feel to you like you understand something even if you don't understand it so what is understanding there's a really useful literature from the philosophy of science about what the nature of understanding is and a little bit of background most philosophers have been really interested in the nature of knowledge which tends to be understood as being sure about individual facts and a lot of philosophy of size people say like that's not what we actually care about we care about something more important we are about understanding which is something broader which is AK about a relationship between facts so you're sharing some theories out there Katherine elegans says understanding is grasping a coherent relationship and not just seeing the individual facts but how they fit together right John come on vague says that understanding is facility with the terrain so it's not just having individual facts but able to move between the parts and the ability to accommodate new information and Michael's Trevon says also when you have understanding you can communicate it right you can get that understanding to other people so here's the phenomena of understanding so if we want to be hostile minted belief manipulators is what we want to do we want a Mac create a system that maximizes the feeling of things sliding into place and the coherent network maximizes the feeling of seeing connections maximizes the capacity to generate some kind of explanation and maximizes the sense of communicability make sense so and we evil manipulators even those are kind of hard to do have an advantage people who are try to get genuine understanding have to actually pay attention to what's true we don't right we can engineer our system for like maximum the maximum feeling of clarity they say they actually comes from my mother adapted a little bit my mother used to tell me that I liked fast food more than home-cooked food but that's because the fast food people didn't care about my health so they just maximize deliciousness but if you care about health you can't maximize deliciousness that's the same idea right the make sense if I care about truth I have to balance things if I just want to get you on my side I can just make it maximally brain delicious right make sense I can pursue the scent of understanding without actually pursuing real understanding so here's one obvious case study I'm not going to talk about this lot because I think it's super obvious a Rush Limbaugh style echo chamber that has a conspiracy theory maximizes this sense right if you have a conspiracy theory it explains everything in terms of one great global conflict in which all these people are out to get Rush Limbaugh right then you can explain all these complexities in the world and it's not just that you have a place for them you have a robust explanation usually like the reason there's this disagreement it's not mysterious it's because the New York Times is in the grip of this cabal of people that are really they're run by Hillary Clinton right so you can give explanations and typically in conspiracy theories everything is connected right and if you make instead of a thousand different forces going on in the world but one central force going on the world you can have the feeling of it creating explanations for everything and notice that the conspiracy theories that you find in Rush Limbaugh type circles it's really easy to generate explanations right so make sense so I think it's really obvious here that echo chambers create this feeling of clarity and get you stop thinking about the theory okay I wouldn't do to all the hate this for a second because you all probably think you're not believing in one of these and then we'll give you something else that I think you may be part of that I'm certainly part of here's another example of false clarity I think Bureau traffic metrics indicators and quantitative assessments also give you the same false sense of clarity so there's this fabulous book from Theodore Porter who's a history of the culture of quantification and you all should read this this book is like this felt to me like an unlocking of everything that was interesting and important about the world right now he's a book called trusted numbers and he says that there are two ways of knowing qualitative and quantitative he thinks that are both important but he thinks we've radically overvalued quantitative ways of knowing right and here's this explanation he says when you get it transition from qualitative to quantitative ways of knowing what happens is qualitative ways of knowing tend to be rich nuanced and sensitive and quantified ways of knowing strip out richness of information nuance and context Civic sensitivity and make information that's portable and comprehensible in radically different places right so here's an example this is I think my single favorite example ever you know in England we you used to measure land in the hide get rid of the hide a hide is the amount of land required to feed an average family and keep them alive okay notice that hides can you feel this now notice that hides are radically sensitive and changing to local environments right a hide near a river is really small hide in the field is medium-sized a hide in a desert is large it's also sensitive to fluctuations so a field this particular part of the woods might be flushed most years but we know one out of every ten years it drops to nothing and a family would die right so a hide measures this but in order to govern a hot the only people that who know who the hide is are the people who live there right central bureaucracies can't figure out what hides our so when we start getting centralized governments and we need to manage land from the center we move towards the acre notice the acre loses all functional sensitivity right loses all the rich contextual knowledge loses the most maybe important thing to know but what it is is it's easily aggregate able by anyone right anyone can build a map with it anyone can use it sänger salary engelmayer has another fabulous book which is basically an extension of that to the future it's called the seductions of quantification she was a part of several UN teams that made various you know things like the Human Development Index that assigned a number at every country based on its degree of human quality she was on the Commission's for this and then she turned an anthropologists eye to trying to figure out how and when these metrics got made and why they got made and one of the things she found was once you get these rankings they get stuck in everybody's brain and everyone just starts using them right the thing they care the most about is the place of their country on the rankings she's amazing definition she says indicators and metrics this is her definition of the idea present a false front of authority by hiding the complexity and subjectivity of their manufacture right what it is creative metric is you have a thousand pieces of information that you interpreted and what you output is here's a single number right and it's vastly powerful because do once you have that number you can create a ranking right you can see exactly where everything stands so I think it's helpful here too for me I always think about this in terms of the educational system because I work in education and I'm not sure about you but in my educational system we're getting a march short quantification more and more every year the newest March is that every budgetary request from my requirement has to be justified by quantitative data about how we'll help student success or student success is defined only as graduation rate okay so I want you to imagine for the moment the difference in a GPA system in education and one based on qualitative imagine I someone told me that GPA is an American term so what work happens in America is everything it's a letter grade a minus B plus B and each of those is translated to a number so you know an a is a 4.0 a B is a 3.0 and then we have a number right and then you can advertise so the the GPA stands for a grade point average which is the average of all your scores in all your classes so everyone comes out of college with the GPA do you do you have something equivalent here I was told the GPA is like weirdly American anyway so when you come out of college you have a GPA let's think that represents your entire college experience right same with high school when you apply the most important thing is your GPA so imagine the difference in a GPA system and education and one based entirely on qualitative evaluation right the qualitative evaluations you can make rich and context-sensitive I could tell you student what they needed to work on what they should be proud of I could keep in mind what they wanted to do in their life right but that couldn't be aggregated couldn't be over viewed by administrators right I couldn't summarize it quickly if I was an admissions officer so we need GPAs instead so what GPAs give you is easy cognitive facility a sense of power and understanding right because what you can do with them is you can aggravate a student's lifetime performance into a single number you would activate a school districts performance into a single number you would set cutoff points for eligibility so if you look at how GPAs get sorted right if a bunch of people are applying to one school that school will just organize them in a spreadsheet by GPA and assign an arbitrary cutoff and then just not look at anyone below that arbitrary cutoff right that gives you an enormous sense of understanding you can manipulate the element order them and see them connections instantly and instantly rank things there's this incredibly interesting book called the engines of anxiety by sociologist named Wendy Iceland and another sociology saint-michel Sauter and it's you know the US News and World Report's from rankings of law schools so there's a ranking of law schools by number it charts what happened to legal education when that ranking showed up and the answer is it destroyed it and the profession may be right so what are the things she thinks and I think this is super interesting sorry mate this this is an important example so um she says before that every school had a mission statement and the mission statement tented goal and a lot of different schools could have different goals right one school could be we're gonna create top corporate lawyers another school could be we're gonna create lawyers engage with the community right and with good moral standing engines of it when the US News & World Report comes along the things that cares the most about are entering GPA and LSAT scores of the students and legal employment rates when the students leave right so what happens is any student that admits people on any other basis drops in the rankings so they say when this came around it created an enormous value realignment where law schools were no longer allowed to have different missions with different values right instead they had to conform to whatever values were behind the rankings or they would lose students and lose funds and the reason the US News and World Report is so satisfying is so catchy is before it students had a really hard time figuring out what school they would go to right but now instead there's this easy legible ranking that presents an authoritative as finalized as understood simple ranking like this okay so so here's the worried US News & World Report rankings get students to stop considering their own values in thinking about the wide variety of values of law schools through a show of false clarity a ranking so here's the idea I want you to concentrate on going forward as I as I've been packing your brain with weird stuff quantitative systems produce a sensation of clarity by discarding information in nuanced and focusing on only some features that make sense so if you cut away a lot of the nuance then it's much easier to stack up data together and aggregate it you get a feeling of power but only when you discard a lot of information so to sum up part 1 the sense of clarity can be used and thinking it's a useful point of vulnerability for Thought hacking and there are two methods oversimplifying explanations and oversimplifying numerical representations of complex data right I just want to be clear here I'm not saying numbers are bad are not saying science is bad I'm saying some things are easily measured and some things like educational equality are really hard to measure but we're like too attracted to hyper simplified measures of it that's part 1 part 2 what you've been waiting for porn so this is a from a paper I co-authored with Becca Williams there's also an by the way I'll tell you later we also wrote a New York Times about it you can read so we were really interested in what started as I will admit a dumb drunk Facebook conversation between me and Becca on someone else's thread about the definite you know philosophers love to define things and we were drunk and we were just trying to find porn in the generic sense this means - there's this new use of porn food part real-estate porn closet porn my wife loves this let me give you some examples food porn my wife loves this site called things neatly organized calm and whenever she feels stressed is it just like thousands of pictures like that and everyone on the site calls an organization porn poverty porn while we were reading the paper a comedian Saturday Night Live was like I'm sick and tired of all this impeachment porn and we were like we know what you mean right so there's this new sense of porn that we're all using we all know what it means what's the definition so we borrow from Michael Ray's definition of sexual pornography here's the actual definition I'm not going to read this I just want to show it to you so you can be impressed by how technical it is but the short version is Michael Raines definition of sexual pornography is that to use something as sexual pornography is to look at that thing's represented sexual content for immediate sexual gratification without interest in personal intimacy that make sense so we thought that sounds right let's generalize it here's our account of a generic porn you use something as ex porn for any X when you use a representation of X for immediate gratification freed from the typical attending costs and consequences of actually engaging with the real thing like tense so food porn you get immediate gratification from a representation of food that the cost and effort and calories of actually eating it right real estate porn which is really common in the west coast okay kids know we can afford anything probably London - I'm sure London tears real estate getting the immediate gratification of representations of real estate without the cost and effort of buying and maintaining it makes sense okay if that's right we should be able to see new version - porn here's one moral outrage porn moral outrage porn is using a representation of moral outrage for immediate gratification freed from the costs and consequences of actual moral engagement so actual and moral engagement includes trying to get things right trying to be nuanced being attentive to the truth acting on it uh if you can't tell I think most of online life is moral outrage porn does it make sense okay so I want to be really clear every given this talk someone miss read this I'm not saying moral outrage is bad I'm saying moral outrage porn is bad right if someone says sexual pornography is bad they're not saying sex is bad right so I am NOT saying there's this like jonathon hate he's a you know American figure is like there's too much outrage ah like around the world we just need be civil and respect each other I'm not saying that at all I think genuine moral outrage is important it's crucial and it's because real moral outrage is so important that the porn version is particularly terrifying I think the porn version undermines the function of the genuine thing so moral outrage porn involves using your moral outrage for the sake of immediate gratification while avoiding the demands of actually good action this make intuitive sense can you kind of feel like what this would be like and I just want to be clear not all porn is bad right I don't think there's anything wrong with using a representation of food for a meet for immediate gratification but there is something wrong intuitively with using a representation of moral outrage for immediate gratification so what's wrong one way to put it is that there's something wrong about using your moral beliefs for pleasure right that's not what moral beliefs are for let's get this okay so I think the deeper danger is the use of moral outrage porn invites us to instrumentally alter our moral belief systems and moral life for pleasure what this means is instead of trying to get your moral beliefs to match what's morally true or good you move them around to maximize the pleasures of moral outrage for right so how would you change your moral belief system to maximize the pleasure of it if you didn't care about getting things morally right think you would simplify it right if things are complicated you keep having to pause you don't get as much moral outrage and you don't get it with much of that pleasing sense of security and you'd want to surround yourself with input that maximizes the pleasure like obvious portrayals of on the other side and lots of people on your side wrong you on right that would increase the pleasure so using moral outrage porn invites us to and this is a technical term that I'm making up I guess instrumentalize our values and moral beliefs right it asks us to change our moral invites us to change our moral beliefs and simplify them to increase the pleasure we get from engaging with this pornographically so think about the difference if you actually wanted to get your mo beliefs right you'd be sensitive to nuance you'd be open to the possibility that you're wrong if you were just out to maximize the pleasures of moral outrage porn you'd want a simple belief system that was easily triggered and constantly exposure to simplistic portrayals that reinforce the belief system while triggering the pleasurable outrage I I'm just hoping that this is seeming familiar to you from like your experience of looking at people talking online so let me put together parts 1 & 2 let's say you want someone to accept a system and then stop thinking about it you would want to create a belief system that condemned and clear and simplistic terms the other side in a way that made you feel smug and superior and at the same time offer a clear and usable explanation of what everyone else was organized to get your group right if those things go together you got something that's pleasurable and payment for simplifying your moral system and then you get a sense of clarity so you stop worrying with the system that make sense so far hopefully the two parts are fitting okay part 3 we're going to take another tangent hopefully I'll bring it back for you games and gamification so this is okay I'm gonna advertise for site this is my book games agency as art it's coming out from AUP and March 2020 it's a this is seven years of my life in a book and it is about why games are a beautiful new art form and if you understand why you'll think gamification is terrible if you don't wait for the book there's a paper version that's out already yes take a picture by the book the first chapter is available in my website I'll tell you about that later for free so one of the basic ideas is that games for me are temporary environments with temporary goals you entered an alternate environment and you pick up new goals while you're there right I mean there's this basic idea my wife and I love each other and then we finished dinner will you pull out a board game then we tried to kill each other for an hour and it's really nice and then we put it away again so I got into this project because I was teaching some philosophy of art about games and I read all the stuff about games as an art form and I didn't believe any of it because it all talked about games as a kind of fiction or a movie and that didn't seem right to me it seemed like that was just the application of an old theory so I went looking for a theory about the artistic medium of games so what's the meat like you know oil is the medium oil paint watercolors the medium of water coloring what's the medium of games first thought drools right can't quite be right because this is this is a game this rock this bouldering this is what I do and a lot of what makes the game good is not just the rules or the physical actuality right this is another this whole talk is just me excuse to talk about this example this is an indie game called pew pew pew pew pew pew pew pew pew you control this you're moving this way when you fire your jet you go up and then when you're not firing you drop and you shoot forward and you have to blow things out of your way or dodge them you don't play with a controller you play two people together with microphones one controls the jet pack by going and the other controls the laser but going give you and the hardest part is you start to laugh wildly because you're supposed to play onstage in front of an audience and you can't make the noise you like mouth dries from spit anyway and that's obviously what's really important there is the ambience of the room the microphone sensitivity how dry your mouth is right it's not just the rules is it the obstacles that's not quite it either I think reiner knizia who is a great german board game designer had it closest when he said this a scoring system creates the players and motivations in the game right so one way to put it is the game designer creates the goals of your in-game self in the book the way I put it is the medium of games is agency that make sense the game designer creates abilities and they create goals they give you a different agent and you enter into it and you find out you find out oh in this game I'm cooperating with all my friends to comprehend emic in this one you figure out oh I'm on this team and we're trying to be that team right you find out new goals and you find out what abilities you have so I'm gonna try to connect it to you now okay one of the reasons I think this is so appealing is because life is terrible in a lot of ways and games aren't terrible in a particular way I think one of the ways that life makes us suffer the most is that what we're trying to do is often very unclear right values are unclear there're STUV all it's unclear what the right values are second their application is often unclear third there's like a thousand values right and it's hard to trade them off and it's hard see how they balance and then there's a billion other values from other people so how the hell do we build a society in games there's one goal its quantified you know exactly where you stand you know that everyone else is pursuing the same goal by the same means that makes sense so I think life is kind of this existential value torture and games give us a little bit of oasis from that make sense we're a little bit you know exactly what the point is and the reason is you're offered a simplified quantified goal that tells you exactly how well you're doing and that it's easily applicable to the world right other pieces are ting to come together a little bit okay so games tell you what to care about and we've developed this enormous capacity anyone who's played gay to do this thing where you read a sheet of paper and it tells you that something is worth points and you just care about that right so my worry is that can be good in games but when we do that on Twitter we destroy society okay in games I think it's okay because the games the goals and games are temporary disposable right the point don't connect to anything else in a lot of games the world is temporary but in gamification you simplify goals in actual life so gamification here's my worry gamification threatens to change the target when you gamify something in the world like education or Twitter when you give people points for engaging in public discourse right you might suddenly instead of carrying out getting things right in public discourse you might suddenly just care about doing whatever is to get you the most points and we know what gets you the most points on Twitter it's a moral outrage content with moral outrage gets four times as much attention and spread as any other content well measured result so consider a phenomenon I've been thinking about this lot call it value capture value capture is what happens when you have rich and subtle values you're placed in a social institutional system that offered you simplify offers you simplified typically quantified versions of those values and then everything turns to crap here are some examples academic assessment grades retweet numbers Facebook Likes numerable wines scoring Rotten Fitbit numbers rotten tomato scores maybe money so like going into school interested in learning but coming out caring about your GPA or the ranking of your law school or starting to exercise for your health but then just getting obsessed with maximizing your Fitbit step numbers I just saw someone nudge their partner or your BMI trying to control your diet for health beginning obsessed and said with lowering your BMI and weight starting to tweet for the sake of truth justice and conversation and ending up just doing whatever makes you go viral the most those are all value capture instances by the way I just want to say a lot of the philosophy I do is very personal and I'm a former game addict and I can just feel the same systems and Twitter just being like I'm gonna level up I'm gonna pass into a thousands of followers now that's sorry so notice in these cases there's a game scoring mechanism we're offering you a clear quantified measure of some dimension and you tend to become motivated to change your values towards it because it makes the activity of pursuit clearer and crisper and more pleasurable like a game but to do that you have to simplify your values so Twitter I've already talked about this enough you understand why I think Twitter is like a gamification of public discourse so gamification also threatens to get us to instrumentalize our values right if you simplify your values and you start caring about something attached to points you'll feel more like a game and it'll be more pleasurable and things will be more essentially clear then you actually have to cope with the hideous existential nuance of the value horror of the pluralistic world so another way to put it in games we put value blinders on let's make sense like in games you just care about you don't care about anything else stop getting points right we're trained in games to just care about whatever it is that gives you points you care about victory conditions and nothing else students have a game that just gives you relief from horror but if I was a belief belief manipulator I'd want I'd want you to have value blinders on right I'd want you just to care about the thing that helped my side and I wouldn't want you to pay attention the other stuff and an easy way to do that would be to attach the values I wanted you to have to a quantified point like system right okay so let's put them all together if you want a system to build a belief system you're a truth manipulator a belief an if you later and you want to build a system to get it to spread and stick to people here's what you want to give it easy explanations for as many phenomena as possible that's false clarity a simplified moral system that condemns everybody on the outside that's an echo chamber you want to gamify it so people will focus on a narrow goal and get pleasure for doing so that gives you the clarity of numbers connect to the pleasure of games and then you put it all in an echo chamber here's another way put it and I'll be honest with you I've been thinking about this a lot and I came up with this way of putting it today at 10 a.m. while drinking a cider it's jon's so we'll see if this works there's one very food spattered page all these systems reinforce and encourage narrowness and moral vision right you just they get you to care about a small number of things and they try to make it in a very near honest is sticky right if you're open in nuance and you see the world in all its complicated ways you have lots of counter evidence and you have to revise your values if you have this trainee explainable well engineered system that sufficiently close to the outside they'll be trapped in it right it's kind of a belief system trap so so here's how it works moral outrage porn gives you pleasure for adopting a simple system numbers hide nuance and make the measured thing most salient right gamification gives people pleasure for adopting the quantified and simplified goals and echo chambers dismiss complexify a new ideas from the outside if you put these together this feels to me like my online experience this feels to me like what I'm seeing in the world so when you put these all together seductive clarity moral outrage porn gamification and echo chambers what you get is systems that give you pleasure as a reward for adopting the moral and value systems and systems that are designed to be sticky by narrowing your vision so you just focus on what the system care once you to care about and nothing else their techniques were engineering narrow-mindedness thanks [Applause]