Dr. Wolfram Schultz: Dopamine Neurons & Reward Processing Explained

[00:00:02]it's a great honor to be joined today by Dr Wolfram Schultz he's a professor of neuroscience at the University of Cambridge a fellow of the Royal Society and one of the world's experts in the study of dopamine neurons and their role in reward processing Wolfram welcome to the nature and nurture podcast okay hi we were talking a little bit about reward processing off camera and it was so exciting that I wanted to wait to get to the recording before uh we finished explaining but you said you started saying reward processing is present in all animals a yes yeah the answer is yes a you find it in anywhere in troph and you find it in Ronin you find it in all all kinds of wher you look find be there's a reason why find it and the reason is how we Define reward reward is reward is what you need to live right reward is what you go after you're accumulating it you you're going fors you approach it and you learn from that to get more right so so so it's the basis of the survival of the individual Gene care so it to stay um an animal or a gene carrier who doesn't have the decent reward system will NE either not survive or will sh and then the genes of that gen carrier disappear and so so I take a a a very as you see simple view of the reward system as being a key um contributor to evolutionary selection or Fitness revolutionary Fitness right so that means that that as our brains have evolved for millions and millions of years from simple brains these simple brains already had her water systems and and we take this over and nature it invents new things but does not reinvent things normally it builds on existing things that means we have still our ancient reward system that we need for the first dang War to to suck the milk for the mother's Lio but our cord is not up to up to scratch at that point of course so so then we when you want to comp um uh constrain our reward seeking by not going for the biggest reward or nothing else or something if you want to think about reward we need a bit more than just simple reward approach generating system we need like in in mels we need a decent cortex and that cortex as you were asking earlier that cortex takes time to mature age I don't know where do it start at age two age three people say the preo corticis is mature before age 25 in human so you have a problem here you have reward system that makes you seek reward and you have a cognitive system that kind of makes balance your reward seeking but that ctive system comes in online little later as your reward seeking system of course it doesn't mature on on the first day when you're 16 years old or something it just matures gradually so you have a lot of competition between the Ade drive to get any as much reward as you can get and the control that remor seeking Behavior by far more complicated systems like the portex which of course has far more complicated neurom Machinery if you just look at the neurons and the ddes and everything in the cortex from compared to let's say the dopamine neurons which are kind of a very basic rewards there a should say the dop neuros by themselves are very complicated as well probably as complicated as the as the cortical neuros but they're just much less and they're much less connected to to the brain that compared to the cortex much much more straightforward Machinery despite all its complexity there's a related line of argument here by affective neuroscientists people like Antonio deazi that Consciousness boils down to emotion and reward and Punishment approach and avoidance positive or negative veillance are these all the same systems when we say reward versus punishment is that fundamentally the same thing as positive or negative affect oh the reward system and the avoidable system is not the same in fact the do Comm system is not involed avoidance in the larger sense it will be involved of not going to smaller Wars when larger Wars are available but they're not involved in running away from pay you would have another avoidance system and it's probably not as simple as the dop and reward system the suring system it certainly involves the Amala with famous fear responses to the Amala but the Amala itself is way bigger structure than the doag and I I'm not sure for what else you need you probably need more for avoidance than just the where you can stimulate the do and the anal go for the stimulation again so so the dopamine excitation is enough to make an animal go for reward that is the by definition the reward that you try to get something you try to get again and that is the DU simulation right um now when it comes to emotion well I I think emotion is one aspect of reward positive emotion um i' say pleasure it's an immediate sensation subjective sensation um that goes for if you have it for longer in the state of happiness is same right become gradually happy because you have repeated me so once you know all this you may develop a desire not only to come happy but also to find something that makes you happy like if you like salad you want to try out a salad you do a salad again you feel really happy because it's very nice feeling to eat salad at least for me and then you desire the salad again now of course when you experience the salad you feel pleasure but the question is probably easy speculation that a lot lot of stuff we do when we go after rewards we actually seeking pleasure from the rewards but if you're really hungry you know and and you find something to eat I'm not sure it's all pleasure that drives it's literally the sense of deprivation you don't have enough sugar in your blood maybe maybe sensing somewhere in and not subjectively not consciously but you might somewhere sensing that you're lacking uh amino acids in your in your blood or Pat in the blood or something um so I I think pleasure is a good karma denominator that would explain why you are seeking rewards but I'm not sure it's explains everything the salad is a really nice example because it's not only the reward of ingesting something nutritious but there's this abstract thought of I'm being healthy this is so good for my long-term Health whereas if you ate a sugary doughnut in the moment physiologically the sugar would trigger a bunch of dopamine spiking as well right but then definit so would it be right to think about there's maybe subcortical unconscious activation directly in response to something physiological like the amount of sugar and then there's a separate cortical system about your metacognition how you actually think about feeling bad about eating the doughnut or feeling good about eating the salad yeah I I don't I don't do that dichotomy between subcortical unconscious and coral conscious processing I understand that's in the light of many people I have I I'm experimental neuroscientist I I want to measure things how can I measure Consciousness I mean I can measure conscious States like being awake or sleeping but how I can I measure Consciousness other than asking the people how they feel about it what they what they feel or something I I don't know I I think I'd ra I'd rather not reply to that question uh and I see that you trying to make these de cous um uh definitions I don't know okay that's probably not wrong but I just I just don't feel comfortable to kind of say yes that's what I need well ignore the dichotomy what is different about the type of say dopamine firing when you ingest something sweet versus when you're thinking about something very abstract and gratifying like sticking to your diet that's for we don't know because we don't know dopamine neurons in humans we only know dopamine neurons in in animals and then we cannot say whether they're Dreaming or something or not dreaming or or imagining Rewards or something this is something that I'm thinking I like to do in humans there are in certain situations neuronal recordings possible in humans um maybe less in dopamine neurons but in the Mika and in fact I have a plan to I try right now um in the next couple years to to work with people that have experience in recordings from humans in certain diseases or reasons of exploring their diseased Brain before they undergo an operation that's a perfectly Neal ethical um approach and in that case you can tell the humans to think about the reward so imagine you are thirsty it's uh hot outside hundreds Farenheit um you haven't you didn't drink anything for 3 4 hours and somebody comes with a cold glass of water a glass of cold water so imagine what would happen to your reward neurs we know there are reward neurons outside of to neur like or so and I would dearly like to know at least those rewards I can look like at what they would be doing in these situations and then I could compare them directly with a the same individual person receiving a TR of water maybe not in in h fire might environment because they would be in the hospital but there some kind of situation which we can compare between the actual reception of reward and imagine reception for reward I would not be surprised if at least in the Amala which is a fairly ctive structure that you would want maybe the dopam as well I wouldn't be surprised although the dopam are more basic they're more they're basic almost brain they're more basic VI far less complicated and complex far sophisticated difficult to say if you ask for do difficult to say if you ask for AIG and Orit F reward NEOS reward NE right in the Bas of Gambia I could imagine that LE thousand them would be active both with the actual receptor of w and with the imagination of we're that all neurons are fairly simple they're binary they either fire or don't fire what does it mean for one type of neuron to be simpler or more complex than another oh well no the the firing or the not firing that relates to to a Time base of a millisecond millisecond action potential has duration one to five milliseconds and uh and within that duration the actual ra will occur or will not so that's a binary event but what we're looking at when when a an animal sees a reward like a drum of Cho um there are more of these action potential per unit of time and unit of time could be something like 50 milliseconds or so so if there are more or less neurons within this interval 5 milliseconds then you would say the neuron PES more unless theur has a stronger or weaker response and then you had a second part of the question about complexity so if it's firing more is that because it's simpler and it's easier to activate or the other way around no no no my complexity I mean the uh the connection with other neurons um the the then Dres receive information from axons from other neurons and these can be completely different neurons they could come from the cortex they could come from subcortical areas or they could come just come from and neurons and there could be many sinapsis there could be less sinapsis the syap can be effective that means if there's an action one that goes on an a into a the PO soal post neuron will also have an action or there may be which would be more complex only very small effects of an action po that comes in and you need like five different action the five Action potentials of different inputs to produce lose synoptic action potential so that's what I mean by complexity more the complexity of the of the information processing at the syapse at the neurons and do the neurons have always the same sinapsis do they have different sinapsis do they have many dendroides do they have few dendroides are the same dendroid are the dendroid always contacted by the same axles from other r or are they completely mixed up that's what I need complexity nothing to do with the actual potential itself when it fires in responsible reward and dopamine neurons are simply wired compared to other parts of the brain I don't know I just sent that yeah I realized that um we know there are about 35 different inputs to do from different areas and that is very complex no no but you see the Doos still come in layers like the cortex when you look at the these old anatomical slides from the 197 80s with these beautiful diagrams from the Cort that's a far more complicated structure and the so say kind of basically a bunch of neurons in a half a million neurons each side of the human the cortex have I no 50 billion neurons on each hemisphere in the humans so there are more neurons there are more layers of structures far more different different types of neurons it's a totally different system the cortex compared to the dopi the most human Neuroscience research that I'm familiar with is neuroimaging where you're looking at huge swats of brain area light up and it could be millions of neurons and it sounds like the type of research you're referring to is invasive research looking at individual neurons firing that can only be done in animals what do you get out of that more fine grained approach that you can't do in neuroimaging yeah well the processing unit of the brain is the neuron it's not quite like the maybe it's like the m in physics or in chemistry it's the basic thing if you take that neuron apart then then you don't have a function in brain anymore and and if you record for 10,000 neurons at the same time you get some signal you don't know if all 10,000 EUR produce a signal or maybe only 500 could you good to death signal then may be mixed neurons in between some neur to reward some neur to movement or something not the do neur but the cortex totally the cortex together quite quite a mixture in different areas different mixtures of neurons that do W move sensory perception stimuli everything so if you want to understand the information processing of the brain if you want to understand how the brain process information you want to look at the the elements that are doing the information processing and that is the neuro you can split it down you can even say it's the action potential it's a syapse or it's the neurotransmitter release that is splitting it down in further but we don't have a means a simple means to measure those with a few exception we can measure doing person will be released but you cannot measure all these other molecules all at the same time you can measure and this pretty good approximation of R activity the the human enriching can look at our questions they are see they want to see if a part A gra structure is involved in something but how the individuals do this like take take a computer apart in the old days the computer had a couple microprocessors and then a lot of memory and a few other chips and you could take a probe electronic probe like from tectronics you can fix it to one of the connection you can see what kind of signal you seen on these this um on these wires that go in and out of these individual processing units nowadays computers are all in one one big micro processor one whole thing you cannot get in there but in the old days you could go with the electric drone and see what Mr siup and I'm actually my talks I often show pictures like that and you see all kinds of different signals that's what the neurons are do right you don't see that in human Imaging you have a totally different question you're asking human Imaging you ask does the freefor cortex um store memory or something is it involved in memory storage the pre some area of pre you're not asking what is the individual processor the neuro doing in this situation right earlier you mentioned an example of reward where you're dehydrated in the desert you get to quench your thirst and that's super rewarding and this seems like a good place to introduce the idea of prediction error or dopamine dips where you could drink the same amount of water and physiologically it's the exact same but it's that Baseline state of being deprived that makes it more rewarding than if you drink it when you're kind of fine and you don't really need it uh no I think there are two different things here what you're referring to is satiety if I'm in the desert and didn't drink for a while I'm certainly very thirsty if I just drank a pint of water i n with thirsty and I'm I would say I'm water SED my reward value goes down with satiety right I'm not going to drink another pint of water that means the water has lost its reward value because unsated on that water right it's got nothing to do with prediction got nothing to do with predic SH the do predic is a totally different thing if there is a reward that is predicted and the reward urse and let's say h let's say it's stra of water it has been perfectly predicted at that moment to occur then you do in your soul fire this is making me realize that I don't understand satiety very well because I think I was conceptualizing it as though you're satiated when you know neurons that are expecting some reward are misfiring and the reward signal no longer comes through the same way no I I think we would like to separate Society from expectation of predictions no no I I I think we we're going the wrong the wrong here no no um do neurons are sensitive to satiety um uh well the evidence is actually not terribly strong I would think there are sensitive to society from all I know about other Warriors where the evidence is much better like Vex well but that got nothing to do with the war ER the reward prediction error uh assumes two things it assumes that there is a prediction um and I he got nothing to do with prediction well I you you could say I'm s then I predict I get more reward no I I don't think so I I think kind of have the question I I I think I at least I would like at the to separate entirety from prediction to avoid say something wrong the prediction on the dop neurons has two two connotations the first thing something is predicted and that can be that should be in quantita matter that means you should not just say you're getting some to drink but you should say you get a glass of wine yourself or a glass of water stay outside of alcohol and and the second thing is specific for the do reward it got to be predicted in time which is over overlooked or often overlooked it got to happen at a particular moment so if you are in the p and you're thirsty and you go to the bar and say can I get a CL just plain what I'll BR you be after and they got get you the water and need turns around and he he uh cools you little water in the glass and gives you the glass I think the prediction occurs so slowly there's no particular time something would occur you probably would not get much of a over response if however you thirsty and uh you go in the p and the guy turns around a whole PL right in front of you that's of water I'm tell you don't like NE with bananas because it's a totally unpredictable event and it is unpredicted in time because you're not predicting that that guy is turning around and give you a glass of what right in front of your mouth right so we can forget the um the temporal aspect of the prediction for the moment because it makes things complicated but it's often Overlook then people come to wrong conclusions and we can prevent these wrong conclusions in our discussion um by me being careful paying attention but um what I want to say is important thing with the is a prediction so if you know you're going to get a reward at a certain time and you actually get that reward your nothing absolutely nothing if that reward is better than you think so for you know we have birthday you buy in in a piece of birthday cake you pretty much know your birthday cakes for the last several years because you had birthday before but suddenly you buy that pie of birthday cake and it's so much better than anything you had expected that drives the dopamine neur nuts they go they get really excited you get 80 90% doers that immediately start firing four five Action potentials within the first 200 milliseconds ex that do normally would not do if that birthday cake is much less or you Nast the brother has put some Vier into your birthday King you get a huge so depression at the moment you bite in it because you predicting the cake as a certain quality taste and everything and something it's terrible right so get a negative Pro with each other that's what do and these predictions we are constantly making unconsciously you're making them I don't know if they're conscious or not conscious as I said I can't do that I cannot say that in animals yes we always make pred our brain makes predictions to the point of where we don't even need to know need to that word conscious everything you do right now our discussion is using predictions I'm predicting where head will be headed directed may I talk to you you will not turn right or left because you have not done past 10 minutes so my brain does these predictions without talking without I'm thinking about it that is and and the predictions for reward are done by do and that's why I say I think that're done worth it unconsciously they're just what youing predictions um these neurons this is called havean learning if you ask me about it I I might recall them then of course it would become yes here when we're talking about reward or that positive or negative affect being linked to how well your predictions are matched or mismatched is that the same thing as consumatory reward where it's immediately in the moment as opposed to the type of reward that's more long-term progressing towards a goal about the journey rather than the destination very interesting um I think the longterm reward has a prediction so you understand correctly your graduate student or undergraduate graduate graduate right so when you had your a letter of acceptance for your rad studies your laboratory or whatever um your do ear were going through because it may have been Prett but you weren't very sure if it's risky and it turns out to be actually true you get a POS prediction error right if it's risky cut 50/50 or whatever you get a big work share well you get that because you get acceptance for your PhD but you know what it really means is your cred studies are leading you to a later more satisfactory life at least that's what be hope so down the line you will have a lot of nice encounters and your stent your friends and everything uh you will you will pass through maybe on laor that are interesting and later you will have a nice profession and so forth so this a whole chain of events that starts in that case at the moment at which you receive that NW and that is this is a soal temporal difference reinforcement learning model that assesses the value over a whole state over a long state in this case over several years right you may actually want to work with the bank where you get more money and and when you when you get a prediction when you a letter that you accepted the bank worn a lot more money than you're doing right now you may like that a lot but if you think about the long-term consequences that your salary in 10 years will not be more than bit higher than right now whereas when you're do your studies you would have much High much higher salary in 10 years Al course money is not everything just taken it as a as a reward so so the idea is the the immediate reward triggers the response but if that immediate reward is part of a longer term beneficial situation that is also rewarding then you get a much bigger reward response is that beneficial situations is it true that depression is a failure to properly inhibit this Cascade where for example you get a bad grade on an exam so there's a negative prediction error there and there's a hit of negative affect and most people will just take that and say okay I'll try harder next time everything will be fine but in depression it could Cascade up this level of abstraction of this means I'm going to fail my program that means I won't be able to get a job that means I won't be able to support myself my life is ruined yes so what's Su question whether depression represents something like a failure to inhibit that prediction error Cascade yes yes yes I think what you're saying is that the prediction error wants what the background of I don't know what you call it state of attitude or something that can be optimistic oh the hell I mean I get next time I'm going to do it better or the other ises I'm to de count no that is true the great CH of forms on the background State attitude whatever you call it you press being one of them and then we have different consequences the same reward Bri sh has totally different consequences depending on what how it is used afterwards right you can use the armor put a nail in the wall I can use a hammer to smash a window because you lost your key this seems to imply that our predictions are all nested and even something very proximal like whether you're not you get a drink of water in front of you in this indirect way it's connected to am I going to be able to meet my homeostatic needs and live longer and you know then all your plans in the future are in some indirect way attached to that absolutely absolutely all these best everything you're saying everything you're seeing Falls comes onto a background in the background and then it gets evaluated well take the other situation right you talk to somebody and they tell you something and you say oh yeah this is really interesting I have an even better idea so the purse is the same meage saying I have a better idea to the person who mean oh yeah let me hear it I'll do it or oh what the guy says I'm idiot you know to the opposite reaction to exactly the same verbal information everything we do every information we do Falls onto a background and it is interpreted or used in a way is compatible with that background the background decides often what you do with that information that comes in right the same information coming in is used in totally different ways so we not aware of that we we blur out on oh we could do that and the person is really mad actually depressed and you don't know why what have I done turns out the person is very sensitive and thinks everything you comment on is something the person has done wrong it happens to everybody every day well you want to be careful what you're saying you don't want to to hurt anybody even though the message this is totally neutral that extreme negative sensitivity again it sounds like depression and in the context of depression you don't hear much about dopamine you hear mostly about serotonin how are these systems similar or different do they work together no no no I want have pull back um the depression is a state um and the state is supposedly caused or associated with true little seron it has never been claimed much that it is associated with the lower tone l l doam but it has been said that depressed people appreciate rewards less or at least use less lose use rewards less as encouragement that means the do system is producing the reward they in the brain it is not involved in the state that defines depression this reduced effective state do supposedly not involved in that that's more Cal to some extent not and nor an effort um but the dopamine if one of the many many symptoms of depression is to not Val do rewards very much then yes then the dopamine reward CLE is underappreciated in depression that does not mean that dopamine is involved in depression it's simply that a brain signal that comes from the Lo urance is not properly used in depress depression does not take account enough properly of the rewards come from d and depression takes it not into proberbly into many other signals they come from many other brain area right the brain areas including do are producing Cs and the depressed mind is not processing in a I don't say properly man but in in a manner that's encouraging to for the behavior of the individual for does serotonin modulate that dopamine signaling absolutely yeah but that may not mean much in the brain many neurotransmitter systems mon each other yeah the the there are some indication that seroni process reward other indications that their process movement serotone neurons are not as a not as not a system like do really 80% Pro one another hetus man admittedly a process reward seron neur do different things different J C neur seem to do entirely different things there are very good studies absolute excellent scientists that come to totally different um neural signals in serotonia neuros and that means something that doesn't mean mean that the people make a mistake but they mean that the system is very complicated I'm trying to piece together a story in my head about the timeline of these different types of predictions it sounds like when we're talking about relatively short-term outcomes like the food that you're getting in front of you or the grade that you receive on this test that's more dopaminergic and if you're thinking about something very abstract and long-term like life satisfaction I hear serotonin involved in that more so first question is that I know it's another dichotomy but it is there some truth to that and then secondly if so is there some sort of translation between a shorter term dopamine action to longer term serotonin action um I'm not sure about the life action andon what we can is there is an idea that the lack of Serotonin produces this involv in depression which means that you need a decent serotonin level to not have depression that's at least the common understanding and I'm not entirely sure whether every psychiatrist agrees with that see but um the evidence that serotonin is involved in longterm satisfaction in an active manner not just by taking seron in way and you have no more life satisfaction but in an active manner like you increase C tone that people are more satisfi with their life I don't know I have not heard but I'm I don't know everything I have not heard all that stuff certain of I fi but I would be surprised if long-term satisfaction is involved this is primarily based on seron I think it's probably more a cognitive process where you have you have to have very good cognitive functions to understand your world make good decisions and as a consequence of that your life satisfaction increases I think there are some studies that corate life satisfaction with um particular neurons in the in a very high Association context between I believe it's between the insula and the or Vortex or the frontal the frontal temporal Junction or something it seems that the number of particular in this area is created with life satisfaction again I think this is a rather straightforward result which do help protect the complexity of Life SCI faction but it needs a hint for what I think it's probably more the quality of Association cortex that deter life satisfaction and these modulator systems like serotonin dopamine or so they may have specific roles in that if you have a bad damine system you're not appreciating your rewards very much you're bound to become depressor because you're not getting the satisfaction but the role of do itself in depression is by no means it established anyway other than by a few serious results or something it's interesting that these top down effects can see to improve our mood but only to a limited extent so you can frame things in a more optimistic way and that can improve your outlook but if you start telling yourself just delies lies or delusions of grandor like I'm a God incarnate and like you say all of these amazing things about yourself having superpowers and if they were true you'd probably see massive dopamine firing but very quickly reality seems to put those in check so you can come up with any cognitive delusion but your more basic processes would seem to inhibit those thoughts maybe unless you have some form of Psychopathology is that right um I I think General yes but yeah but you can use your Liber thinking um to be more cral to help people to kind of be the first smiling face of the day for for people they smile back at you and you few something so by the liate decision to be Pro social you get social response back that can make you happy I'm not saying that all social makes you happy all this I I don't want to generalize here but I think you can you if you think I'm happy man that will not no I I I agree with that but I think you can do some action that that can kind of indirectly make you happy there's always these stories of people that are very religious and they wake up and you say oh you God will provide me a happy day they go outside these nighted people and they say every start back at me today is that a wonderful day and all it took is to believe in God and believe that God is doing something good for you right that works it is more complicated than than just saying yeah I'll be a happy I'm a happy person that that's wrong but but the kind of make a positive Pro to Li including social life which is Big denominator of your of your happiness uh I think it can work but I'm not saying that is the result of a good less good projects or FR projects if you have a bad project you mess things up you become frustrated and frustration is certainly not producing happiness so if you are always wonder own foood you always irrational you always do what everybody else would never do you're not going to become you can be productive you can discover new things but you're not going to become very happy you become very happy when your stuff works when you understand things when you don't make tic mistakes that you curse yourself afterwards for two days that is what makes you happy and for that you can have the cortex but just use the cortex I'm happy today I don't think that will work that's not enough people talk about the human cortex being unique in its ability to plan for the future and engage in metacognition and I know it's hard to study Consciousness in animals but looking at the basic neurobiology the way the neurons are working in other animals and if that's conserved across humans can we get any hint of understanding whether animals are doing this type of metacognitive prediction as well I wouldn't be surprised maybe not the honey be may maybe not the fruit fly but uh but I think manels with their complex cortex but you see there is a transition somewhere in the the line of evolution of the cortex the only animal that really have very well organized Spees hum I mean monkeys or not monkeys Apes can learn to communicate verbally but what I'm hearing always is that're not very spontaneous they can be drilled they can be trained conditioned to use words hundreds of words but they're not apparent not very spontaneous that they suddenly come up with something so I'm not sure I think these animals have Consciousness yes no problem what level of conscious I don't know as over debate whether they have metacognition is in my opinion a similar question as whether they have language or not that means abstract language in which we can write about our own thoughts this is metac contion right you can think about what you're thinking um I don't know if if Apes have that which are probably maybe elephants maybe whales maybe dolphins that are the most developed mans I don't know but the the example the Lang of this fantastic language capacity that all humans have makes me wonder how much we can simply copy paste all our cognitive facility capacities on to other mountains they surely have lot of comp capacity probably more than we think uh and they probably have more Consciousness oh yes I'm pretty much persuaded I've been working with Mar for 40 years 45 years I'm persuaded they have Consciousness they can think they can think they can sit in the corner think and do something but much much less than apes and much much less than humans would you say that it's similar to human children's level of cognitive complexity maybe I don't know you human children are amazing they're fantastic I I'm not sure sure whether the human infant is comparable to a gany for example I don't I'm not so sure of it I think it's the human infant is immature adult it's not just a different adult CH is a different adult than the human adult it's not the same thing I I this is not very sir I'm just speculating and I'm trying to answer your question rested as honestly as I can without with some vation without going but some do we know about how the reward system develops in humans from infancy and I'm specifically thinking about you know some people say that you have all of your neurons organized at Birth and it's more about pruning and organization but it's not like new neurons are being born right so do those dopamine neurons look mature from the start I don't know I don't I don't and is that because we can't do the type of invasive cellular research in humans no we would we wouldn't need to as sad as it is brains de babies um to do that to look at that I'm sure this has be looked at I just don't know that how much that differentiated School back to the reward system these infants must have a reward system otherwise they would never s mid but milk for them is a reward the behavior they're showing from the first day on when they unus is a typical reward seeking Behavior I don't know that goes to the do system and is small humans but I would be surprised if it would not because I don't know another system be it efficient so I think do neurons are probably more or less option they may not have all the dent C Dev they probably don't have the dent well developed they may not be as sophisticated as they are in the adults but I would be surprised if they would not have a f reward but they do a roed error or simp respond to rewards I don't know either because the reward we to share is something that comes from the neural connectivity from the puts to the do and they are probably not totally they're certainly not totally matur and I don't know if all fun so I don't know if they would have a reor prediction that he cannot test because we would need to Liv in B and look at the brain that is not there's no way of doing that you mentioned at the beginning that at least compared to other areas of the cortex the reward system matures relatively early and I've heard some arguments that during adolescence is when it matures and specifically that you see increased levels or maybe increased receptor density specifically during adolescence and then dopamine receptor density or sensitivity decreases into adulthood is that consistent with your understanding yeah I I I would not be surprised because beginning the brain remodels it itself it produces con that it loses the connection of course needs um vles and receptors and everything yes so that that would that would work perfectly well well yeah no no I would have no problem with that now why would it go up during adolescence and then later decrease during adulthood well the general the general idea is this is something general for all neur connections there are more connections formed that will be stabilized in longer longer life and the one of the ideas I heard was that there is no particular building plan for the Euro Connections in the genes that the man is simply connect with whatever he can find with and only that that will be used properly and will be necessary Will Survive and the rest will disappear so it's not a plan of a onetoone connection genetically which would need a lot of keys it's more like it man connect what you can find with and we'll see what will work and the rest will disappear it's much much easier principle I don't know if that's the only explanation of this transient pruning or transient higher number of sinuses that disappear but that's a least one argument I've heard what was interesting to me about this unique argument for reward sensitivity during adolescence is that unlike regular pruning where you might have more neurons during childhood that then pruned during adulthood and the remaining connections get stronger it wasn't just that linear decrease it was smaller number of connections during childhood that raises during adolescence and then later prunes in adulthood so it's kind of like an inverted u-shaped argument yeah yeah yeah yeah the the connections will not form what within the first years they're not there when the baby is Bor the dendroid in the frontal portex are not very mature when the baby is Bor and the gender a lot mature they don't attract much axons the axons for l L but then maybe too many form and they disappear afterward so it's it's a it's a inverted new shape yes because bir not everything is is there that needs to be there I mean the neurons themselves are there but they still have axles that grow slowly then make contact some contacts will be stabilized other contacts will disappear and that takes days weeks the front what takes 10 two two 20 or so year 25 years or something like that until you get a stabilized Network we talked briefly about how increased reward sensitivity during adolescence could relate to increased risk-taking and some people especially evolutionary psychologists make an argument that this is adaptive so there's something about that period of sexual maturation where you want to be especially sensitive to reward and to exploration and it's adaptive that it later decreases and then another possibility could be something like it's just a spandrel something about the brain development needs there to be this peak with increased sensitivity in order to have that pruning but it's not specifically selected for in an Adaptive way no I I don't know go well I think I don't know maybe you're too sens I'm not sure you're s too sensitive to reward you have a totally different attitude as an adolescent the risk taking is only a part of your exploratory behavior um if you don't explore things you get stuck with what you have and that is fine when you 60 years years old or 70 years old you're happy with everything you got but when you're 10 years old you want to explore the world you want to see what's going on if you already know at 10 years old what you want my God what a lot what a boring life you want to expand your ideas you want to try out things you will try out 50 things on one of which will work will be fantastic with the other 49 you can forget that's called that that happens that happens also with irrational behavior when you do lots of goed mistakes well you you do a lot of damage you lose a lot of time but you might inadvertently run into a situation that is extremely powerful helpful that one situation makes worse all these other 49 Mystics you're doing so you want to be expansive when you're young you want try out everything children play well the children's play is not because they're stupid the children's play is a very smart program to understand explore the world and and see what you can do in the world if You observe people children what any time between the age of when they become a bit more independent like one or two years old they start working with the all war compared to a six-year-old they never SW well I'm exaggerated it's for much less than children and everybody go nuts with these children then always take their elect train apart instead of let around circles well they take the electric train apart because they want to understand the engine the electric motor and and they take it apart of course they can put it back together and the whole thing falls apart because they don't have exp but in doing so they Le ultimately how to approach a problem maybe even how to understand electric motor but at least how to approach a problem not to just sit there and say no nothing can be done I'm I'm doomed no they go and say let go try let's try this let's try this word one of these only one out of 50 or so may succeed and then learn something and the children they seem to do that 15 hours a day or something all the time it's amazing we don't we we don't have time we don't have the interest we don't even have the energy and that's because they want to build up their world and they want to build up their brain because whatever they doing that training that whatever you do changes your brain our discussion changes your and my brain right whatever anybody does change their bra the children w train their bra I mean not consciously but it happens children that or that individuals that as young children try to drain their brain have a higher chance of survival in evolution and reproduce an evolution those that don't train their friend you don't see them anymore they have disappeared because they got lost during their lifetime you knowless millions of years to to work there's a machine learning analogy here of you could get stuck in a local Optima and then any gradient descent in any direction is going to ex decrease prediction even though there could be a a global Optimum somewhere else behind the mountain is a higher save yes uhuh so you need noise or exploration and that's social mammals occasionally jump high and you'll all the mountain something so eratic chaining one out of 50 times we jump a bit High you look over the mountain you see oh there's a higher there so let's vot there yes exactly if you think about play especially in Social mammals spend so much time playing it it can be confusing from a survival of the fittest perspective because you might think why are you running around wasting energy isn't it better to conserve energy for survival's sake but the argument here is that through play you're practicing and calibrating your reward system and learning to engage socially well so social engagement is far more complicated do reward system right you need a lot of Cortex understand other people the mind understand understand other lines the do they just respond to a reward somewhere some neurons detect this is a reward and not just the neutral sensory stup okay you need to have that but once you have that the neurons just respond passively social inter action is unbelievably complicated if you think about it and H cortex and if it's complicated you need a lot much ler training if I claim that the do preneurs are almost doing their job from birth on maybe not exactly as we see it in the adult anim almost there the the human cortex is no way social comp competent at even at H high or tenant right so that they takes a lot more training to comp cord is much more complicated take more training the situation is more complex you need a more complex Machinery to do that we've been talking about reward very generally but there are different facets of reward so there would be reward sensitivity or sensitivity to magnitude of different reward or different qualitative rewards like monetary or food or sex are these all using different brain systems or is it one domain General dopamine reward system the the dopamine system is still J but as you mentioned different rewards there are C sensor system that makes a differ that make difference you need to identify the reward and then you to compute the value of that reward the value of that reward start with the do but the neur Do Not distinguish between I I don't know about sex but I would think so don't dis insects and trop virtues they only say how good it is that is called the value or the subjective value that's all they do you need other brain areas mostly the cortex that least in hum that tell you what is the actual reward so in that sense dopamine neurons they're more affective they're generating that positive or negative emotional experience and the cortex is interpreting things in a cognitive contextual way I not sure involved in effect responses I don't know I would think be more than I m I think it's more basic it's the value the value is how much a reward is good for you how much it helps you to survive today and how much it helps you to propagate your genes that is to have maing progress that's the value you you infer you cannot measure that value that's famous value but you can infer from choices would you choose more frequently or would you prefer has by definition a higher value that's the definition of of value that's what do they do the value and they do it in this reward prediction manner right they don't tell you what the reward is and they don't tell you what you needed for they only care how good it is or how bad it is I mean not bad in the in the sense of being aversive but it can be very low value very high value very low value I would say it's bad and very high value is very good but they're not interested in paying off here could you explain the functional organization of the dopamine system across different brain areas I often see it as a circle beginning at the brain stem and projecting upwards to the cortex and then the cortex projects back to the brain stem in this cyclical process well it's a bit more than bra Stone it's it's in the mid bread well it's the basis of the brain the doo are have their cell bodies in the midbrain and the ventral lower part midbrain and uh and project the axels into the stri to the basic mostly and and cod nule and to a good extent also into the cortex and these areas feedback the the reward systems when you look at them completely are totally reciprocally interconnect that means the reward neur the do reward neur cortex the cortex reward neurons go to do and many other cortex neurons also go do neur are also connected some do project to Theon and project back into do the same thing with the St do go to Str and go back to Str so and and then of course you Vortex reward are connected to am to the I'm not sure they all reward and they go both directions so you have a little Network between do and these are the four principle rewards that are interested in reward without anything else but all reward structures except the dup neuros are also having other neuros that are identifying the different types of reward like juice reward fuit reward and sexual is the sarabellum involved at all no maybe there are some indications that are there's a plenty of neurons that are sensitive to re in many brain areas but when you look closely at that they're often in association with WS or with sensory stimulation for CH PR CeX very famously has neurons with reward are driven by reward but they also driven with movs and could have a combination of excitation due to reward and to move the same thing in some n Str the same thing other in the stratum are purely reward so this what I called the reward syst contains neurons that are only interest reward but reward influen a large number of of brain areas as and as I said of in association with other things like Ste ey or but for war has processed in large areas in in the rain a prediction in particular seems to rely on memory so I imagine there's also some hippocampus involvement there uh I don't know that could be but um prediction you see predictive responses do neurons and will this prediction come from maybe from the of from the St the the hyppocampus I don't know I mean the prediction are built in a very in a second or so whether they hress I'm sure there are NE Congress that follow predictions in the second R but but to me it's more alter memory thing or more an organizational memory stal memory and things like this I think the minute by minute or second by second prediction that we see in all super is is also coming from stral and I don't what is happening probably some studies that show prediction responses in the avula but how representative that is for the whole I don't have a full picture at the neuronal level when predictions change what is happening what does it mean for a neuron to have a prediction it almost implies that it has memory is that stored somewhere implies that the neuron receives information from memory the neuron responds to a stimulus that predicts the reward all right let's I I wave banana to you and that banana predicts reward even if you don't get it right now Banana is a two reward predicting stimulus so your memory somewhere understands this is bana I've seen that I know it I can give name and everything right that's not what the D do but the D will respond to that banana if I suddenly show you where my hand I'm trying to CL my hand my hand comes and let's say my hand is a banana suddenly that would fire a do and the memory that this is a f is not in the do but oh I don't know what if it's in the do system value that this prediction has for you or for me well yeah I I think that's basically it it's a it evoke it has access to memory this St neur has access to memory is influenced by that memory I don't know I don't think the do is storing this memory content I don't know I'm getting I'm not sure what I'm saying now getting too complicated I don't know let's talk about addiction and the role dopamine plays in that I hear a lot of evolutionary arguments in particular that there's a mismatch between our ancestral environment and our modern day environment where for example foods high in fat and sugar used to be very rare and therefore evolv to be rewarding because of their high energy content but nowadays they're so abundant and they're bad for us and a lot of people struggle with that or other things that are designed to be addictive like gambling or video games or smartphone usage all these things you can name that we get sucked into and people often use this terminology of it's hijacking our dopamine system does that hijacking phrase sound right I the in as of all over millions of years most of the time until 100 years ago if you take the example of P we were doing hard labor 80 90% of us were farmers then they became Factory workers well they had the energy consumption of 6 to 8,000 calories a day they could not get the calories proteins and from sugars from carbohydrates they needed to eat fat so and that is how evolu a brain that does not recognize fat as calorie containing provided that braing has no chance of survive and individuals with such a brain would have disar they don't exist anymore so what we see now is animals and humans in particular our humans as we're talking of them that For Whom the reward system is particularly tuned to F not only the do I don't know that about do speci we haven't looked at that but the current humans place in very very high reward value into fat and it probably take many many generations in which hum must do not need that fat anymore until the Val fat decreases for the brain Ur right you would need a mutations basically in the brain in which the Hat loses its attraction for the reward and evolution goes very slowly 80 Generations away from Jesus we basically have the same brain as Jesus they're about eight mutations per human generation and many of these mutation have have actually no consequence others are terrible and others produce a progress in the evolution of our G right so we are extremely slow in changing ourselves our brains and everything and if the Industrial Revolution or whatever we call it the Modern Life has land so fast it really outpaces our evolutionary speed and of course the reward neuros or the the the rewards the F reward is detected by do not only by dup so the dup are by whole need probably the most consistent and strongest reward neurs in the brain so if you want to focus on D neurs by saying hij Jack in the dopam system it's probably not wrong but it's not only dops that determine that Pat is a good reward you certainly have overit photic you have aula there are neurons in a that specifically detect so our reward system is is um not adaptive to our current situation and it can only adapt through evolutionary process basically through mutation gene mutations of which there are eight per generation and the generation 25 30 years or something right so the gambling thing sorry the the addiction to iPhone or telephone and and and in media is still a little different language I can explain but this this beity is at least what I come comes to mind here when I talk about the attractiveness or the reward of value of P that makes sense and and how about the technology addiction yeah uh the funny thing is deurs do not only respond to reward puzzles a lot of people and creates a lot of confusion the dopamine reward response is preceded U by 50 to a couple hundred milliseconds by an unspecific resp that only detects the simulus without having identified whether it's a reward or not so they respond very well to aers stimul but they only detect the stimulus as such like this is a stimulus that when I'm doing this what I'm foring similar right now may not be strong enough to fire D but the do respond to ver stimul but not to the divers they simply respond to detection or to the appearance of the stimulus and as soon as the the neurons have understood it's not a reward the response decrease immediately goes into inhibition often but this initial response if that is strong enough I mean it will procure release dopam it is a dopamine response and this release dopy can have the same effects as reward it means plasticity and learning so if you have a strong enough neutral simulus like something you see on the Tik Tok or so that could fire that would most certainly fire do and it can have a picity effect and if you have too much dopam firing then you get um addiction so you could have in some brains not at all other brains who could have addiction I don't get addiction to that stuff I don't get I I I walk in the US or Puerto Rico into hot tells have slot machines cck directly next to the reception and I I don't react I'm unar it and I'm not interested and colleagues that come up with me they immediately jump to these machines so people have different brains not everybody gets addicted to Tik Tok some people get addicted to Tik Tok some people don't even look at Tik Tok or they look at it and don't care about right so the brains are different between everybody but in some cases the sub suspect the doen release from this very intense simul can produce dop release the dopam release can then have produce addiction to not everybody gets addicted to cocaine either you know brain got to be sensitive or have particular strong dopamine responses or strong receptor responses from dopam I'm thinking about the prediction in my brain going on looking at your camera right now whereas the lighting has dimmed down your body is appearing to you know kind of just appear back and forth like like a phantom and yes so it was actually very illustrative because even though the raw sensory stimuli are strange it's something that I'm so easily able to contextualize as this is just the zoom call your body is there statically and not changing it's just a perceptual illusion based on the lighting but illusion is perception of the light of the the color or the intensity of my face for example yeah it's um it's half past 4 local time that outside can see is is there any dopamine neuron action involved there where for example there are perceptual neurons perceiving you and expecting a certain outline and if that outline is blurry or fidgety then there's any prediction error involved there if if the outline is a reward and if that the pzy outline looks like nest for reward yes you would have a negative position but in a subtle way now now that the camera is well lit and showing showing your face in HD again that's rewarding in its own way so if my face excuse me if my face were an apple and you couldn't see it very clearly you w't sure it was an apple I go switch on the line suddenly the app feels like that you would have heart R that's precisely what I meant by the temporal sensitivity but suddenly the the Apple becomes visible you see oh here's my Apple within half a second it's clear then you would have a nice do response yes and this domain generality tomato then you would have negative do response as soon as you understand it how tomato mhm and it's domain General enough that whether that's food or a social reward like seeing an individual's smile all of this is activating essentially the same system the the soldal reward yes there have be S on soal reward but it's more like reward was given to another oh sorry I sorry I forgot the data I I'm not sure it fire Sil MOS um the the the pH soter award has never been tested on we were we were looking that we were analyzing um we we showing movies King Kong Jones to monkeys it turned out M Perce as reward the reactions are typical reward but you never got off reporting from it was the idea initially but the they do so little it's they have so little reward value that you would get enough data out that was a confusion but I have award for monkeys watching a movie like Indiana Jones do they also show the same effects like let's say there's an action chase scene and in humans you'd see increased heart rate or something like that do the monkeys respond the same way I don't know but I would think so they were they had to touch a lever and as soon as we release the lever was over oh yes they were GL to the lever uhhuh yes yes they like that but when we had make the movement to touch a lever they did not more than five or so in 15 seconds and that's not enough for us for to get data MH well from would you like to close talking about your recent work on risk-taking that you were mentioning earlier yeah oh well you're around interest because the way assessing subjective reward value and the idea is the reward small and large reward are not NE a large rewards not necess better than a small reward for examp when you sat it then people refer to that as subjective reward value that means the subjective value goes down with saity we mentioned that earlier but the physical amount of rewards saying so if you want clim that do response to a reward that is Meaningful to the animal you need to show that the cing subjective reward and not just reward amount because the amount as my STI example those is not always important for the what is important is how the animal values itself that reward if the animal is thirsty the reward is higher value and one way to show that is through risk and well it's actually more complicated and there is a very good definition can about subjective value which is called utility and you can get utility you can have a mathematically defined utility because we're working with very precise Behavior here very precise neural recordings you want to have precision and we want to have mathematical definitions of subjective value and one of those is utility the formal economic math ically defined utility utility functions and to assess those functions um one way one of the two main ways is to go through risk because risk modifies the subjected value right if you risk a voider and I present you the same reward as risky reward or safe reward you get the same mean reward you will prefer the safe reward you would even prefer less safe reward than the risky reward because you hate that risk right so risk if finds subjective War value very so we talked about that we we we worked on that with economists the Dos are C the subjective reward Val very much they C all utility FS so this is related to Conan's loss aversion bias where whether it's risk or whether it's loss we're more sensitive to that even if the expected utility even if we're making making an irrational decision looking at expected utility well the irrational decision the definition of rationality here probably hinges on the reward amount and when you are risk avoider you are going for Less reward if it is under the condition risky so you are you are foregoing larger Awards you can call the irrational I would not call it Ira the definition of tionality in economics is subjective the maximization of subjective value and vis a risk avoider going for smaller Awards is still maximizing utility the the subjective value so I would not call risk avoiding or risk-seeking irrational behavior and um the other aspect in irrational behavior is interesting to bring it the other aspect is that when when we Define rationality by maximization of subjective value not objective value when we Define it by maximization of subjective value then your subjective value and my subjective value may not be the same most likely they the dark mind economics is new utility function my f cannot be compared you cannot feel how much I put utility into an apple or something well I dve how much utility just as a famous economist once said you do not feel my tooth AE you can only feel see that I have a tooth AE but you don't really feel you don't know how intense it is right so the irrationality is same thing what seems utility maximization for me may not be utility maximization for you and then you say I'm irrational because I'm not maximizing utility in the same way you would do that right so this is last conclusion my rationality and your rationality are different and so saying you are or that I am irrational from your point of view it's just basically me you're not behaving the way I'm not behaving the way you would behave or vice versa but to say that as irrational I'm not so sure do we have I'm overly emotional and you say emotional is the opposite rational yes then you can say it I wouldn't do that do we have a sense of the source of these individual differences or the directionality where for example you have genetic differences in sensitivity of certain dopamine neurons and that leads you to take a more reward sensitive attitude as opposed to something like you consciously adopt a different strategy and then because of neuroplasticity that changes the way your brain responds I don't know I I think that is a fair question and the ask probably yes I just don't know it and I don't know it's everybody knows well come back to my gambling experiment I pass at SL machine in a hotel reception and I have nothing my colleag goes there and gets addicted and pulls this thing off all the time right in his opinion I'm irational because I'm not gambling I'm not going to my best subjective value for him the gambling has highest subjective value he loves that risk okay I don't so so for him I'm IR for me that guy is or that person is heral because he he or she is wasting all their money with the Stu machines which you know they usually waste their money right so why one is that why is this other person G I'm not ging I'm sure I have different numbers and concentration of receptors Su receptor garate all receptors different numbers of neurons Hing different connections not totally different find connections yes brains are different I love this work because of how it applies at literally all levels of analysis from understanding neural firing all the way up to understanding abstract economic motivations underlying decision-making and really everything in between the the visual psychophysicists long debated whether the red for you is the same red as for me right I mean we see a particular wavelength and we call that R do we really see it the same way it's it's impossible it's just like my and your toothache we cannot compare it and the same thing is the subjective value and then to say that well it looks like you rather stier than mine therefore you iral I'm not iral I don't think I don't think I to do that do you think there's an attraction to those types of philosophical questions that can never be answered because it's a never-ending Journey yes NeverEnding Journeys are very interesting you always have to new argument you see 3 years ago I would have not had that argument about irrationality but I've thought about it and I have a son neuroeconomics as well he's been discussing this and I've discussed it with other people this this your truth EG is not my truth EG comes from a very famous economist who told about somebody else and you know things develop I I don't I think these intract problems including the famous Consciousness will people us busy for a long time and will be very interesting and they're all very rewarding not consequences if you think somebody else is irrational you're more bound to kind of vir and if you respect it be a simply different than you are then you probably respected that and you may not wait to war against people like that thank you very much for your time wol friend okay well it was fun talk to you it's very similar you as well very reward in look