Moral Constraint and Choice: Neuroethics Frontiers with Fiery Cushman.

[00:00:06]our series poses is the question that keeps me up at night but I'm really good to work on during the day the question is what contributions does neuroscience make to other fields have been query I do neuroscience for sure most of what I do is just a neural experimental psychology and it's not always clear to me what contribution even the neuroscience that I'm doing makes to this the more frankly psychological topics that I spend most of my time on let alone the other areas that I enjoy learning about like philosophy or law and my life as a lawyer so we spend a lot of time thinking about those intersections and so to have the opportunity to put together a kind of a new talk on new material and very little of this is going to be about the work that I do but more of a survey of the field losing this question what can we draw from neuroscience has just been already rewarding for me and I'm sure through the question-and-answer of the discussions afterwards there will be many many more rewards don't you imagine that you desperately need money by tomorrow say $100 $1000 and I don't know why maybe it's kind of like Guys and Dolls or Nathan Detroit to make a movie and upon me he has a thousand tonight for some reason that you can just get your hands on this money it would make a huge difference in their life like maybe you're late for a big job interview you just need the money to get into a cab and then you see somebody drop their wallet here's the perfect opportunity of the wallet keep the money for yourself so this is obviously gonna set up a conflict in your mind between your desire for money and your desire to be normal right whatever your moral concerns are they're going to be in conflict and neuroscientists and psychologists obviously invested a lot of effort into trying to understand both of these different kinds of value their psychological role the neural basis neuroscientists in particular have just made extraordinary progress I think this is one of the biggest success stories in the behavioral sciences in understanding the mechanisms in the brain and the computations that they perform that allow us to make would make think of its financial decisions sort of self-interested decisions about value and a in the neurosciences is that to the extent that we can understand how people make moral decisions at all it is basically by treating morality as just another good to be maximized I mean it's different in the sense that it's connected to other people what's good for you when it's moral concern is ultimately the point it has for another person or how it is that they think about you so that that's different than just maximizing your own goods but at the end of the day it's the same type of neural systems the same type of computations that we use to make choices just for ourselves and choices that have and this really stands in contrast to psychology there's this gap for a kind of a square peg round hole relation because the dominant in psychological research on morality has been that there's something very different about our moral considerations as compared to a bar stricly credential or personal or financial considerations and I think that probably ordinary people's intuitions and a lot of philosophical research are on the side of the psychologists that is if you just think to yourself what does it feel like to me to need that money to get to the job interview and to steal that money person it feels like there's really big differences between those two things like like you want the money but you could leave the money but there's something about the moral concern which is just a rule that you can't violate that puts an absolute constraint on their behavior so what I want to do in this clock critically is to help you to see vividly this mismatch between on the one hand our intuitions and psychological research on the other hand neuroscientific research so that you can experience the same dilemma that keeps me up at night and then I'm going to talk you through what are some promissory notes new research directions and my lab and I can see these same directions and many other people's research programs as well where we're trying to kind of have our cake and eat it too and hold on to what's right about both of these insights but to start to narrow or eliminate the gap in the approaches so to do that I'm gonna start by talking about the neuro scientific evidence and as I mentioned I think in general the last 20 years of research in the neuroscience of value guided decision-making has been an extraordinary success story and I've Illustrated it here just by pulling out one computational model and one region of the brain this is the straight line that we're seeing right here but this stands in for many many different an attentional models that explain many different parts of the brain in general what these mathematical models do is they translate experience into behavior so in the first instance they learn from experience and then based on what they learn they help to guide the death one of the big breakthroughs that happened in neuroscience was around the middle of the 1990s when computer scientists who've been just trying to design computers to solve these sorts of problems and in fact this particular equation q-learning was derived from approaches in computer science started teaming up with neuroscientist and the neuro scientists realized that this specific algorithms the computer scientists had alighted upon seem to be implemented quite literally implemented in parts of the brain you to track the activation level of a neuron over time and it would track one of the numerical parameters in this equation and it also did a really decent job at predicting behavior so here you have a marriage a precise computational model it provides a good explanation for behavior and it seems to capture the essence of the neural computation that a biological system is performing that trifecta is so rare in behavioral science that's why it's so exciting so I'm going to give you just a couple of examples first it's something that has nothing to do with morality that'll give you a sense for how people have studied these phenomenon and then we'll look at how the insights developed for economic or preventional decision making started to show up when people studied moral phenomena so here's an early study this by Brian Dixon and colleagues back in 2006 they start by bringing people into the lab and asking them to just rate how much they like a variety you have different consumer goods like the ones I'm showing here then they put them in the scanner and in the scanner they're gonna offer these goods for different prices and in the beginning of the experiment with a little person money that they can spend it's enough to buy a few things but not enough to buy everything so participants are making real decisions about how they want to spend the money on the consumer goods that are presented one by one and what they find is that different parts of the brain and seem to be responding to different features of this experiment so in the nucleus accumbens they find a signal that basically seems to capture the ratings that they provided earlier of the value of these goods if they rate agenda highly the nucleus accumbens respond strongly when it sees gender the insula seems to be instead tracking the cost that's associated the financial cost that you would have to pay in order to obtain that vote and then the ventromedial prefrontal cortex this region in the middle here seems to be balancing those costs and benefits against each other in order to converge on the choice of a particular item and they could show that above and beyond the behavioral range that they collected in the beginning they could get extra predictive power in guessing what the participant was going to choose by tracking the neural signal in venture medial prefrontal cortex so I'm gonna introduce a graphical representation of this model which I'm going to use for the rest of the talk so it's important to follow this the idea is that in ventromedial prefrontal cortex this region that I've circled here if you need a thousand dollars by tomorrow you're going to be considering multiple different candidate actions that you could take to try to obtain that and the key thing that ventromedial prefrontal cortex does is to assign in a common currency in some common scale a quantitative value to each one of those different options so that they can be weighed against each other those mathews are being drawn from other parts of your brain on that last slide for instance the nucleus accumbens and the insula and many other brain regions aside and it could be extraordinarily complex processes that these brain regions and antecedent ones are going through in order to generate what ultimately the vmPFC translates into some kind of a common currency the key thing of course about having the common currency is that that allows you to make a unitary decision weighing all these things against each other so as this model was being developed an obvious research direction that many many different labs pursued was to see how well to capture social or moral here for instance is a paper by yamagishi and police in which they have people make generous allocations of money or not you can either keep the money for yourself or give it and you're doing either well other people are watching you or not and what they find is that the nucleus accumbens one of these brain region as I mentioned earlier that tracks the value of consumer goods is sensitive to both of these factors and you observe an interaction between the factors and just the way you might expect namely the value of doing good as we're presented in this brain region increases what other people are watching every sensible idea so it's not a big psychological insight but it's cool to start to get the sense for which brain regions are accomplishing this and what makes it cool is that we know those brain regions are also making contributing to financial decisions here's the study by Jamila Saki and Jason Mitchell a colleague of mine in this study again you're just allocating sums of money between yourself in this case or Jason's roommate and I'm waiting and what they find is that video for example cortex this brain region that it's supposed to help facilitate in all things considered a common currency devaluations goods is sensitive in particular to the degree of fairness of the allocation that you're making and unfair allocations are inequitable choices ones that screw over the other person a lot for just an incremental gain for yourself are associated with activation in the insulin that was the brain region that also responded negatively to the financial cost there is a long history of research dating all the way back to Phineas Gage in neuropsychology asking what are the characteristic patterns of injury to the brain that produce deficits in morality so we know that individuals who like Phineas Gage suffered damage to ventromedial prefrontal cortex tend to have dysregulated behavior they make abnormal judgments in the Charley problem but it's the behavioral deficits that are really much more important than the trolley profit objective mints here's one of the earlier rigorous studies that was done by Andersen colleagues of two different individuals who had suffered damage to ventromedial prefrontal cortex I'm going to focus on this one who was run over by a vehicle at the age of 15 months so the kinds of world deficits that I'm talking about you stole her family children chocolate that frequently led to multiple arrests was verbally and physically abusive to others who lied chronically there was little or no evidence that she experienced empathy or maternal behavior was marked by a dangerous intensity insensitivity did did the infant's needs so clearly she has behavioural normal deficit the key point that I want to make is that individuals with ventromedial prefrontal damage don't just have dysregulated moral behavior they make terrible decisions for themselves just a financial decision it's personal decisions Prudential decisions the same individual was largely unresponsive to physical or verbal punishment engage in early and risky sexual behavior leading to pregnancy at age 18 repeatedly put herself in physical and financial risk just earlier and what you find is if you administer a task where over time you can learn to make wise financial decisions as compared to ordinary individuals who learn to choose from the good deck of cards with brain damage are relatively more likely to make some optimal decisions so I could go on and on and on about the accumulated evidence that this network of brain regions that makes Prudential choices is also implicated moral decision-making let me rather just sum up with a couple of abstracts from I think really thoughtful well-done analyses of you know reviews of the state about our services one from just a couple years ago about rich in rough and fair and they read this suggest this set of evidence suggests beautified mechanism the behavior that may incorporate both social and non several factors now they're pending with me and in the final sentence they say but doesn't it kind of feel like the social moral stuff is a little different and at the end of the paper they try to speculate a little bit on where those differences could be but they're mostly wanted to throw up their hands if they're not finding a lot of differences in the neuroscientific literature similarly my good friend man Young and the student James Duncan right you know at this point there's no neural substrate or system that uniquely supports normal cognition in this review I wouldn't discuss early assumptions with domain specificity of moral neuroscience as well as the subsequent investigations in general now again they hinge at the end and they say we're going to consider the possibility that this isn't the whole story because it just doesn't feel like you can possibly be right but that's where the evidence right now one of the things that I really like that Leanna Jane so you might think the reason that neuroscientists haven't found unique substrates of moral decision-making is because they've never designed experiments to find them in the first place they took as their starting point as a model these Prudential or personal financial decisions and the experience that they designed were never equipped to identify unique moral substrates at all but if anything just the opposite is true if you rewind the tape 20 years ago the first social neuro scientists were all running studies where they were trying to contrast moral and non moral decision-making and I don't think that that set of studies has left a robust legacy of replicable solid results of clear differences and it was only over time with the repeated failures of that contrastive approach that people started to say maybe there isn't a contrast drei vier maybe morality is just a species of generalization now you can obviously tell where my heart lies I've hadn't hoping that we'll find something different something that's unique but I want to point out just how cool it would be if this model works for morality we would have a neuroscientific model of moral decision-making which would be an enormous accomplishment and in and of itself which I think for people who are trying to think about ethical problems in the real world let me draw an example that was pointed out just in one half of one sentence of one of my favorite papers in the literature in Josh Greene so in this paper you know they were they were approaching it from a neuroscientific perspective and they showed that if you do a whole lot of troubling problems where you vary the number of people on the tracks and also the probability that the train is gonna swish that vs. that they get off the tracks you vary magnitude and probability of our classic kind of utility based approach the brain networks that you recover that seem to be tracking magnitude and probability are identical to the ones that people in neuro economics have been finding for financial decision-making for a long time so another brick in the wall it's just ordinary financial decision making at the neuroscience level but then in this in this one half of one sentence of the paper based not on the neuro scientific evidence but on the behavioral data they just remark that when you look at people's choices they beautifully respect the law of diminishing marginal utility this is the mainstay of economic theory is that the first you know Little Debbie new high that you eat it's pretty good but once you've had twenty they don't start to taste so good anymore so they have diminishing bounding to you and what they observe is that the very same thing is true of saving human lives the first human life that you save feels really great but by the time you're saving twenty human lives the marginal difference of being able to save one more life just doesn't feel that compelling anymore now that's got to be a moral mistake right but it makes sense that that principle would be embodied in unreflective automatic intuitive moral decision-making if it's co-opting circuits that originally evolved for foraging for food not for making trade-offs in human lives and so the neuroscientific model that says that there's really these are the very same system that are being included in both cases can help orient us to potential blind spots or errors that and I think that you know that has an important payoff from kind of a climatic standpoint okay that being said as I mentioned there is also wonderful research in the psychological tradition that suggests that perhaps the defining feature of a decision being a moral decision is that it is not based on this kind of domain general utility you don't reduce moral values to just another common currency against this 15 points in favor of that one of the early illustrations that this came from the developmental literature just in three and four year old kids Elliott Ariel drew the distinction between what he called genuine moral violations and conventional violations and it showed that very young children respect the same division so a classic case of a moral violation would be hitting another classmate classic case of a conventional violation my boy who wears a dress to school and so what curiel shows is that even though children distinguish these in the following ways they think it's very bad to harm another person but less bad to break a kind of customary conventional rule they think that if you do violence against a person you must be punished for that it would be an injustice if you were not punished whereas it's not represent in the same way for a conventional education they think that moral violations are universal you say imagine that there's some other place where it's okay any children is it okay for them to do it that way and they say no but if you say imagine there's another place where boys do wear dresses to school is it okay for them to do it that way people say yeah that's pretty much okay and connected to that idea if you say well imagine that you came into school one day are the teachers meant to you on Wednesdays from now on we can hit each other so if you've got someone you want a hit Wednesday kids say no absolutely you can't change the rule just by an authority that you're telling you so what you can do this at the teacher announces that Wednesday is where the other genders clothes day that's okay independent and one of my favorite extensions of this work which is incredibly generative that said about yes they went to children in very very committed communities of religious faith so for instance Amish children Orthodox Jewish children and they said what if God came to earth and said on Wednesdays you can hit people and even then the children persistance say you know in that case not to be wrong because even on Wednesdays you cannot hit a person that is not allowed so the key idea here I mean this isn't exactly the distinction between morality and a financial decision-making but it suggests that moral decisions have this characteristic flavor of being rules that you can't violate under any circumstances and a utility based approach to decision making never has that feature there's always a price there's always some amount of money that God could offer you that would make you hit somebody in the face but the claim here is that that's that's exactly what isn't a moral consideration so fetlock here independent around the same time also john barren and developed similar evidence from adults DoCoMo the sacred values john called the predicted values similar idea here's an example of one of close studies so in this study he described a whole set of different things and said would it be appropriate for someone to spend money to buy myself alone or buy food or to buy a cleaning service health care or legal services not surprisingly everybody thinks that but they said what about spending money to buy some of these organs or if you want to adopt a baby to buy a baby or if you're a baby who wants a parent to buy a parent or my citizenship by sex to buy a trial by jury to buy somebody else to go to prison for you or somebody else to serve anymore for you and what he finds is that people think that it's offensive to even suggest spending money on these things it's not that they're saying that yeah they're just a very high price they're saying it's a category mistake to think that money could ever buy any of these things that generalization holds true across numerous different political persuasions except libertarian it actually turns out that I was putting together this slide yesterday this popped up on my Twitter feed this was all over the news yesterday and I think it's a beautiful example of Phil's insight that people just think it's crazy that in the hospital you could get a bill for holding your child right because holding their child is just not the kind of thing so this idea is I think this is a really intuitive idea I can see about many of your faces that yes you feel this way about morality to philosophers feel this way famously Elizabeth Anscombe once wrote if somebody who really thinks in advance then it is open to question you could ever have a debate over whether an action like procuring a judicial execution of should be excluded from consideration I don't even want to argue with that person they show up rough life in other words she's suggesting that there are certain concerns that are so self-evident and that there is no conversation we ever have no amount of utility that could ever lead a person to to accept for instance killing an innocent person okay so that's the guy hopefully if you have a sense for the gap that exists between the neuroscience and psychology at work and so what I want to be for the rest of the talk is to suggest if you just tend to do theoretical proposal so that how you would go about trying to close that gap I mentioned this review that Christian Ruffin Ernst bear wrote just a couple years ago where they summarized one of their ideas was that maybe this is just a methods available maybe there are moral and non moral decision-making circuits of the brand that are independent mostly but coexist right on top of each other so that if you get damage to one you're gonna get damage to another if you test one in neuroimaging experiment you're gonna find the other right on top of it but it's not because it is literally the same network it's just because there now there's evidence for something like this future you go and you do single unit recordings are recording from just one girl neuronal time for instance there's a group that's found that an ex you'll find some neurons that are relatively more selective for social value and other neurons that are relatively more selective for you know what I'm glossing its financial now there's some evidence for that on the other hand I don't think that this is really what we were hoping for because what we have for these brain regions isn't just a location in space we have a computational model the model tells us in this location in space they are doing something like a utility based evaluation and so the mere fact that there might be a neuron that responds to social value doesn't save you from this paradox if the if the computational role that that neuron plays is still to enter into a utility calculation so here's another idea that I think has occurred to a lot of people it seems like a central idea that and I think this one has to be at least part of the story is just has huge negative value so it looks to us like there's an accent ruler constraint here but that's just because it's so rare that there's something that can override it's incredibly negative value I do think this is part of the story although I think it has some shortcuts but because I think it's part of the story we've been actually trying to build some evolutionary models that would allow us to predict when that type of huge negative value whatever wants so let me just sketch out them in broad strokes the way that this model works we envision a very simple world where we have individuals who are for goods and so for instance if this guy gets an apple that's positive for him he gets resources now of course the girl has a choice she could forage for her own but she also has the possibility of stealing in that home and then you know he could fight back maybe throw a tomato at her her and then keep tally so that they would that's the basic set of action that's better permissible in this world obviously sentence like this have been exhaustively studied an evolutionary game theory an evolutionary game theory you endowed these agents with fixed strategies that they're going to pursue over the entire course of their life always deal never steal try stealing but if you get punished stop stealing each of those is a strategy that's unchanging the approach that we took that was a little different in this project was instead to endow all of the agents with general-purpose utility calculators and then what we allow to evolve is the basic goods in the world that they would assign utility to but then they would have to learn the best strategy and their strategy could change over time depending on how much reward okay so an obvious way to start is that we should make it valuable they should experience positive utility whenever they eat an apple and they should experience negative utility if they get hit in the face with the tomato so we we built that in and that was refer and now the question is are you gonna learn whether to steal or not just by comparing the number of apples you get to the number of tomatoes you get soft with and are you gonna learn to punish by comparing the number of apples you're able to keep and the energy that you have to expend Earling Tomatoes to defend them or will evolution favor assigning intrinsic value like we talked about the sweetness of revenge is that an eight is it we know then that he is the human mind designed to find the experience of seeking revenge speed not something you have to learn has value but it just intrinsically feels that way and similarly you might think that obeying the rules has you know there's a way to like innate goody-two-shoes obeying the rules like that intrinsic positive values for us so we explored this a few different ways one is through a formal model of evolutionary dynamics and another is by actually building agents that instantiated easy 280 based rules and allowing them to play each other in a tournament and then to reproduce an evolve over time these two approaches converge on a comic insight which is that you use learning to decide whether to steal because the constant benefits are immediately if you steal an apples if that's what you would need and if you don't steal unsuccessfully pretty quickly you get time and because those costs and benefits are fairly immediate learning suffice is until you put no intrinsic and Nate wait on stealing or not stealing waiting but for punishment the benefits of punishment are relatively more long-term if we get in a fight the immediate consequences that I feel works I got a little bit of myself in that fight and I'm banking them on a long-term benefit that over time I will convince you to stop stealing from me and the difficulty of that learning problem to a first approximation necessitates that you have an innate bias to find the very act of seeking revenge recordable and this is consistent with experimental evidence if we put people in an experiment it's these people turning red here they're an experiment where they have the opportunity to steal from somebody but every time they steal they're going to get punished and it gets even testing whether it's in the beginning that we're starting out by stealing but pretty quickly they learn I'm getting something for this it's not worth it and they give up these people here in blue these are people who are getting stolen from and no matter how much they punish they're gonna keep getting stolen from we've rigged the experiment so that they will never stop having their goods taken but they never learned the stop position you see no diminishment over time and the degree to which the consistent with the idea that revenge is literally sweet for them that they're not trying to learn works are they are good wired to fight back when somebody steals from them so that's an approach that we've taken to try to develop evolutionary models that can account for where this type of large negative value assignment or positive value assignment to different moral or social actions might come from but as I mentioned I don't think that this account is gonna take you the whole way to closing the gap between the neuroscience and psychological perspectives what amazing look in the trolley problem it's fine to say that there's a large negative value associated with this person diet but that should mean that there's five times as much of a large negative value associated with these people value so just saying we really don't like it when people die isn't going to explain why people very very hesitant to to engage in that action or another way of just about this is to think back to the phenomenon the idea that we just feel like there are certain things that you are not allowed to put a price on it's just that's different than saying that the price is very high if it were just a matter of the price being very high people would read this article because it's an incredible bargain right $40 before the baby are you kidding me sign me up I assigned a much higher price that has a tremendous amount of value to me but of course that's not the response that people have that this is a bargain rather the response that they have is then this is not the kind of thing we never under so that's next approach begins with some of my favorite work on happy and we're gonna see how this connective morality in a second but you're just got to follow me as I describe the essence of what happened is so ordinarily we produce an action because we want to produce a result we lock the door because we want to make things safe inside of our office or outside of our home okay this is an example of goal directed behavior they have a goal in mind I want to secure my life compensating the form an action that accomplishes that goal and it's it's the result that you care about it not the action itself you know intrinsically loved a lot of others but what it means to habit I something to make something a habit is different value actually starts to migrate backwards over time and becomes a sign intrinsically to the action and the way that you would know whether somebody has formed this type of is that you can take away the results but they would still go from the engine for instance your your spouse is still at home so there's no need for you to lock the door but you want get anyway just obviously so the idea is that's because you've started to assign some kind of intrinsic value to the action itself in fact there are circumstances where you'll perform the behavior even when it's exactly the opposite of the result that you want to produce like for instance your keys are still in the house and you know that fact and yet you bought yourself out of that house so those are the signatures of individual response so I said I Molly Crockett Dave Rand a number of different people have been working with variants of this idea have begun to wonder whether similar have a learning mechanism might explain something about morality for instance when I go to Europe I still tip the waitstaff 20 percent I know that this is not the local model and it's not because I really love European weights fast like never once in my life has it occurred to me to just write a check to a European waiter and ship it across the Atlantic the reason that I do it maybe because I have started to assign intrinsic value to the action of tipping 20 percent in a way that is now no longer connected to the results that initially motivated so the idea is maybe something like you think would be immoral habit or for instance if you look at the life of an ordinary developing human the most violent year of their life is is the year between their returning 2 & 3 years old which they spend mostly on the playground like hitting and biting and scratching and pushing other people and they get a lot of negative feedback I mean they get punished but they also get the negative feedback of seeing that this really hurts people's feelings which to most ordinary developing humans feels very bad for them and so they maybe started to internalize this kind of habitual value assignment that says the very and biting and scratching those very acts now feel bad and I wouldn't want to produce them even in an unusual circumstance where it would produce a beneficial result one way that we explore this in the lab is by giving people the disabled and gun and asking them to shoot me in the head while measuring a suite of different physiological responses like their blood pressure and the rate at which the heart is pushing out blood and we find that when we do this and it really freaks people out right so here's for instance that increased above baseline and total peripheral resistance when we talk about getting gun like cold hands are cold and if you're asked to do something you really don't want to do you literally do get cold hands and cold feet it's because of the constriction of your peripheral vascular system and that's what we're measuring in this study and we see a big robust increase when you're asked to shoot somebody in the head with a fake gun I'm assuming this is not because they thought the gun was loaded or then there was any risk that the gun was looking that they had thought there was any risk at all that the gun was loaded that certainly hoped they wouldn't and it's a much bigger effect than you see if you just asked them to watch one person another person there's something about having that really gun in your hand that makes the difference and we think that might have something to do with this kind of even if this is a metaphor something like individual responses value that you've assigned to sensory motor properties of an active self that you can't turn off even when you know that it overrides so that's one idea where moral norms might have the slightly different flavor maybe there are systems that assignment you know actions somehow more powerfully or more not over I doubly for morality than for financial decisions but as you can see this is the point of the argument where it starts to break down a little bit and this is why I think this too is probably even though I think there's something right we're on to some kind of useful I give here it's probably not going to be the whole story so the last thing we've been thinking about in the lab and the most recent set of ideas that's been guiding our research pursues yet another different way to kind of narrow this gap this is an example of a very very difficult financial decision making in neuroscience so you'll give people different cues here represented by Frank bolts and for literally hundreds of trials you click on the different cues and every time you click on them you get some amount of money and you lose some amount of money and over time you're the reason that people do this I mean obviously this isn't what the real world looks like but in the real world we're making choices between things like we just got somebody with two bottles of olive oil and he's had them before and he's trying to make up his mind so did he is the fractals or something like the real world a shortcoming of this approach although it obviously has tremendous value is that in fact the real world looks a little bit more like this by the time you have just four options in front of you and you're carefully trying to decide between those four you have screened out an infinite number of other things you might have been doing I mean there's not literally but there's every other thing in the grocery store you could have gone to theirs every other store you could have gone to theirs every way you can spend your time other than shopping somehow constantly among the infinite set of things we could be doing or actions we could be considering we're focusing our attention on a limited number and the utility calculations that we perform are entering just for that limited number here's a way of graphically representing this on the same scheme that I've been using that is out there in the ether somewhere it's possible millions of different things if we're trying to get money by tomorrow most of which are useless and most of them never enter our heads in the first place there's got to be some mechanism that samples from this infinite set and generates a choice set or a kind of proposal set which is the subset that we are going to consider so is it possible that one of the computational roles that morality plays is to make sure that certain things never get in the choice set in the first place then when you really need money it doesn't even occur to you in the first place to try to get it by stealing from another person or if certain options do with hurt you is there a mechanism that Yanks them out of the to itself so this would hang on to the idea that any action that we do perform had to pass through a utility calculation and if we do a morally good action it came through here we do a morally act and it came through there but in addition to those utility calculations there's some other set of processes that set up the very variables that are enter into the compilation in the first place and that's it that's a way that that moral norms could be operating I came to this set actually what a phrasing I didn't come to this instead of ideas Jonathan Phillips a postdoc in my lab has been doing really rude appalling work connected to these ideas and that's what motivated our president thinking he works on us Batman philosophy and he works on this problem that philosophers call mortality so you can think of muggles roughly as representations of what is or is not possible here's a study that Jonathan Mann years ago inspired by the passage actually a parastatals he said imagine that there's a captain at sea and a storm blows in the ship is going to capsize because it's too heavy and so the captain orders the skipper to throw a cargo overboard to save the ship with the captain wouldn't was the skipper forced to throw the cargo overboard did the captain force the skipper to throw that's appropriate kind of course the skipper that's death law now imagine that there's no cargo differently what there is is the captain's wife so the captain says to the skipper you better for my wife so with the skipper forced to through the white Globe reporter what he finds is that people are are less likely to say that the skipper was forced to throw the wife over for the zombie if you ask did the captain force the skipper to throw the wife will report then they say yes the captain divorce so the captain forced the skipper to throw the wife overboard even though the skipper wasn't forced to throw the wife overboard so there's something really sneaky going on with the way that people are using this word force now Horst is a modal concept the person is forced to do something if there was not a possible alternative modality is all about the representation of what is possible or what is impossible and the idea is that when throwing the cargo overboard is the option here considering we're willing to exclude all other possibilities and say this was the only viable possibility that existed ergo they were forced but when you're throwing the wife overboard suddenly you're you want to get other possibilities in that choice set and so you're saying no he wasn't forced that there was like what reporter could have died himself or Jesus wasn't there something else he could have done right is the basic idea this this basic pattern of judgment is incredibly pronounced in young children so Jonathan Phillips and Paul Newman show that if you ask a young child if it is possible for them to steal a candy bar from the store most of them will tell you it is not possible now do they understand what possibility means you can ask them whether improbable events are possible like is it possible there's a banana on your bed right now and many children will say yes I think it's possible that there's a banana under my bed right now but it is not possible to steal a piece of candy from the store they followed up on this with a study where they said which of the following things would take magic would it take magic to find a banana or better for their anyone that no that wouldn't take magic what about stealing a candy bar from a store yes that would take magic if the kids aren't enough so then we were wondering can we show anything like this in adults well obviously not mean you can if you're asking adult is it possible to steal candy to do but we wonder if we put them under time pressure would we start to see a change so in this study I'm going to show you ratings of possibility to get money by tomorrow and I'm just going to give you a few examples we have a very very large set of different actions that so would it be possible to borrow the money almost everybody thinks that it's possible whether they're making reflective judgments and would it be possible to get that money by tomorrow by growing it on a tree almost everybody thinks that that's impossible whether or they have a long time or they have less but would it be possible to scale well people are a little more reluctant to say that it's possible with this deal although on the hall on the whole they will end up reflection but when you make them perform a seated judgment suddenly they're saying that and so what we think this may reflect is that our automatic intuitive stance is just to try to prevent certain immoral actions from ever making it into utility calculation at the first place and that's a way of preserving the idea that what we do do is the product of agility calculation but then there's something real about the idea that morality places absolute constraints that operate outside elation to synthesize these perspectives from so just to review I suggested three potential ways that we could try to narrow this gap so one of them is probably extremely large values on things another one is the idea that maybe morality involves the assignment of intrinsic value in a way kind of analogous to have a learner and the final one is that morality systematically excludes and I want to suggest even though I as you can tell I don't feel like I have a really good handle on which of any of these is the best explanation if there's anything right about these ideas at all this has important points of contact to other disciplines in other areas of studies I mean if you think of like theoretical philosophy that is not applied ethics but the ordinary practice of moral philosophy of all these kinds of computations Aristotle thought that habits played a really important role in thought that in fact if you were doing a utility based calculation in the world a minute that was not a world choice at all the only way that you could be moral would be to have law like inclusions and exclusions and those different theories of human nature those are those are descriptive theories of human nature on the part of each of these three philosophers in each of their cases it played an important role in determining what they thought he actually moral truths might be the appropriate moral theory so I think there's important philosophical implications but there's also important provisions these different models have very different things to say about how you would take a person with immoral this regulation and fix that would you would this involve like lots of training to people get a new habit or would it involve making suggestions to get them to consider something they haven't considered or is it just hopeless because they were born with the wrong into a new function you know that there's anything to be important consequences for applied ethics as well and so I'm excited to see where the field as a whole