Evolutionary Stable Strategies, Nash Equilibria & Iterated Games

[00:00:00]I was thinking that if the other person prod to steal it doesn't matter whether I chose steal or to split I don't nothing yes so so it's slightly different to the prison dmma is that what you're going to say yeah so here it's fair to like choose split because there is a probability that the other person does not choose split exactly so if you were to look at the payoff Matrix for this it is a little bit different if we call this player A and B and we've got split for deal and suppose our pot of money is M then here you'd have m/2 if you play steel and they play steel you get zero if you play split and they stay steel you get zero and if you steal and they split you get M so this is a little bit different from the prison dilem it's the same oh main principle but really yet the the optimal thing to do is here to uh to steal because if they if they split then you should always deal but if you're able to collaborate in some way then it could potentially all your outcome and this is that's the kind of similarity to the prisons dilemma um the classic prisons dilemma is that they're completely separated and they can't communicate with each other one of the things that goes on in biological population say males and females is they can signal to each other their intentions so and then it comes into question of are your intentions honest is the signal that you're giving out an honest signal um that comes into you know the genetic level it comes into ideas then about linkage so we talked about linkage disequilibrium and this is all about when certain alals or genes are more likely to be associated with each other you expect by random when you have that kind of linkage that can maybe uh help tie together ideas about is your signal honest and is your trait of altruism together so if they're close together they're more likely to appear together and therefore your signal might be honest it's a little bit beyond some of the things we've been doing lately but it the the point is that honest signal becomes very important when there's a chance of of of communicating your intentions yeah when you're talk about signal like you signal signal like signal with each other and those signal each other or can also signal um I'm talking the it doesn't the hawk Dove game I guess you could signal in the hawk Dove game you know are you going to attack or not but then there's a question is that an honest signal that you are attacking it's it's kind of related um but the ideas about honest signaling are slightly more about um yeah I mean ultimately it's all about whether your your intentions that you're displaying whatever those happen to be are true or not or whether you're bluffing so there is a similarity there in terms of the like the do's display but they're just they're just bluffing essentially um and in some sense that's not a true signal um but the dove strategy could also be sort of I'm willing to share this rather than actually display um okay let's move on because yeah just trying to cover some stuff some actual MTH um so today we're going to be talking about evolutionarily stable strategies you don't have to stay sitting next to the the person if uh I don't want you guys to overly socialize it's fine okay so uh yeah the other day we talked a little bit about optimal strategies we talked about this Hawk Dove game as well as the pro dilemma that we've just been through now and we're going to be talking a bit today about ideas around optimal strategies and in particular the idea of an evolutionarily stable strategy or ESS we'll be coming back to this idea of an essess many times throughout the course so definitely pay attention to that um we'll also be talking about Nash equilibria which are related to essss but not quite the same thing so yeah we'll talk about Nash equilibria these ess's um other Concepts today the support of a strategy and also the idea of a repeated game so what you've just done now is just one game one round of a game but you could imagine what happens if you play the game against your your opponent over and over and over again that might alter your strategy because there's a possibility there for things like punishment okay so let's just go back to the hawk Dove game for a second here so this was what was introduced in the last lecture remember the hawk and the dove are uh these are just strategies they're not actual Hawks and dubs in this particular example we can just think of them as being heritable strategies there's one population individuals can play with a certain probability distribution any combination of these strategies and we want to see you know what is the optimal probability distribution um that probability distribution might ultimately be just play this one strategy all the time or it might be some combination of those strategies okay so we our Hawks contest for our resource but potentially get injured um whereas our dobs display they Bluff but then they Retreat if they get attacked and we talked about this payoff Matrix for some values B which is the benefit of that resource and C which is the cost of getting injured we have this pay of Matrix a where a j is the payoff to individual paying strategy I against someone whose paying strategy J so this is your average payoff that you get and in this Matrix here we've got Hawks and doves I'm just going to write H's and D's here to indicate those Hawks and doves so a hawk and Hawk play each other they got a 50/50 chance of uh getting the resource um but on average they will also get injured as well with cost of C so that average payoff is going to be a halftimes by B minus C if a Hawking has a dove then the H gets the benefit B they get the whole resource the dob gets nothing and if there are two dos there's a 50/50 chance that one of the dobs will just back down they're never going to fight over it but one of them wins and one of them loses so on average they get B over two and we showed that if B is greater than C then the hawk is always the best strategy and if B is less than C then the two strategies can coexist at some probability distribution which we found was actually P star is equal to B over C and this P star was the the frequency of the hawk strategy so this is how often the hawk strategy would be played and in that case every individual could be playing that fork strategy but just with probability B over C every time okay so in general we want to know which strategy is optimal in any given game and under a given set of conditions so one thing we can do in game theory is to identify things called Nash equilibria and these are named after John Nash who pioneered Game Theory uh again Game Theory really started out as think about human behaviors especially in like an economic sense but also in like a military sense and a Nash equilibrium is a situation where individ ual who switch to a different strategy cannot increase their payoff okay so you're playing a particular strategy if you switch to a different strategy and you can increase your payoff then it's not a Nash equilibrium if you can increase your payoff if you can't increase your payoff then it is equilibrium So based on our payoff Matrix that we had before a Nash equilibrium has to satisfy a strategy playing against itself a of I Against I is going to be greater than or equal to a of j i and this is true for all J I've got there for J not equal to I in my notes but it's actually for J okay it's pretty simple condition you just have the highest payoff we can see here for example if you if B is greater than C if you play Hawk for example um I like SEC there we go okay if you uh if your opponent sorry if you're playing fork and B is greater than C you can't increase your payoff by switching to play Dove because B is greater than C so this is positive and this is zero so that would be a Nash equilibrium and if you're playing Dove down here if you're playing Dove then oh sorry if your opponent is playing dove then what should you do well you could switch to being Fork so that's not going to ever be doves by itself is never going to be a natur equilibrium okay you can always switch strategy okay the other way of thinking about this is a equilibrium is just a case where a strategy where there is no better response to itself okay so that's all to say the same things now there's a slight variation on this if we actually have a strict inequality so that a i i is strictly greater than AI or Aji sorry and this now is for all J not equal to I then this is a strict Nash equilibrium in otherwise it's the unique best response to itself not only can you not switch strategies and increase your payoff you can't strch switch strategies and have the same payoff so in this case up here if this was was an equal sign if there was some other strategy that you could switch to that would have the same payoff you still not been able to to better it but there is some other response that could also be as good and that gives you a weak Nash equilibrium whereas the condition above gives you a strict Nash equilibrium so question can a mixed strategy be a strict Nash equilibrium no why not op op yeah so if you have a mixed strategy then that is that one way of looking at is is this probability distribution the other is that there is a another set of individuals in the population who always play that strategy and we're just defined by the frequencies of those strategies so in that sense uh more than one strategy will have the same payoff so no more than one strategy that's the same pil okay so what about an evolutionary stable strategy then ESS so ESS is a Nash equilibrium but it just has one additional condition we have that the payoff for I against J some other strategy J is greater than the payoff of that strategy against itself strategy J this is true for all J not equal to I so not that if I is a strict n equilibrium then this condition is automatically satisfied it's the best thing against itself and it's the best thing against anything else so when I plays against J this is going to be better than J against J so what does this additional condition imply for a strategy that is a weak n equilibrium and hint here is consider the case when Strat is right what does this condition tell us so imagine we've got a population that is just composed of one strategy strategy I here and everyone is playing that strategy it could be a mix strategy it could be pure strategy doesn't matter um but suppose then we have some other strategy which is IM mutation a rare mutation or some immigration event where something new comes into the population want to see whether that new thing can invade and replace what is currently back okay so the question here is saying okay well how does that strategy do against itself this new thing that's coming in J if it doesn't do very well if it does worse than the things it's currently dominant in the population then it's not going to be able to spread if it does better then it is going to be able to spread so this is telling you something about whether a rare thing in the population can invade okay so this is basically saying if J is as good so remember a week equilibrium we're saying that J is as good as I so it's uh I just write out again we n equilibrium we're going to have a i i is equal to a uh j i so if J is as good a response to I as I is to itself then our additional condition requires I to be a better response to J and J is what does this mean well this means what I just said this means J cannot invade when R when we got on to the section of the course about adaptive Dynamics which is related to uh Game Theory adaptive Dynamics is also known as evolutionary Invasion analysis it's all about understanding this process of invasion and replacement okay one way you can think about these payoffs or fitnesses is you can think of the the fitness as being like a mountain okay and the peak of the mountain is your ESS that's your highest level of Fitness the question here is saying is there a is there another mountain peak that is as high what we're also saying here is can you actually invade can you can this rare strategy is as good as that same height as your current Peak can actually invade okay so last time we drew some graphs to think about how uh Fitness varied with the frequency of a particular strategy so what I want to do is just illustrate this distinction between nria and 's using the same kind of ideas so imagine if we've got the frequency of strategy I increasing and we've got Fitness okay and we're going to look at the fitness of two different types in two scenarios okay so suppose let's call it AED suppose the fitness of strategy J decreases like this in our first scenario and the fitness of strategy I decreases like that so we'll call this we'll call this w i and this WJ so the first question is strategy I an ash equilibrium from the SC why isn't so J J is this one here is this one here yeah but I has higher Fitness here but I the key thing is the relative Fitness so key thing is whether I is greater than J so you remember our condition up here for a n equilibrium is whether here we're talking about payoffs but We're translating this to the idea of Fitness we're talking about Evolution the the Fitness in this case of strategy I has to be greater than or equal to the fitness of strategy J for all of our strategies and in this case this is for all frequencies of those strategies so in this case here this is an natur equilibrium right because this Fitness is always greater than or equal to this one here okay so there is an ash equilibrium but the question is then is it an ESS well for it to be an ESS we need to have the fitness of this I to be strictly greater than J When J is very very rare and here it's not the equal okay so this is a weak Nash equilibrium I is a weak Nash equilibrium which I'll just abbreviate NE e oh sorry I've I've made a mistake I've got these around the wrong way this is an esss sorry I've looked at my I've Dre my graphs the wrong way around apologies this is an esss this is a weak Nash equilibrium and here's ANS my apologies I was getting ahead of myself and thinking about the next graph this is an ESS because when J is rare sorry as J tries to invade even though they're equal here as soon as you deviate slightly away from one here I is strictly greater thanj they intersect at one so apologies for the confusion there these lines here intersect at one as soon as you move ever so slightly away so there's a tiny bit of this strategy J in the population this red line is strictly below this green line and therefore it can't invade so my apologies for the conclusion there if we it might become a little bit clearer if I switch these lines around and compared to what happens if we have this scenario sure these lines s up okay if we flip these lines around now and we have this scenario the this at this value over here is an ash equilibrium at p is to one this our frequency of I we call P is actually a a equum they have at this point it is better than all other strategies however as soon as this J strategy starts to invade it has higher Fitness and so it can invade Okay so here if you remember maybe the the easier way of thinking about this is the idea of um a resident population then a rare mutant so really what we are we're starting at these points this yellow Dash yellow highlight here okay so ignore everything else the left of these curves for now you start off here both of these cases are we weak as equilibria because I is the best response to itself and is there's no better response to it okay so I J have the same thickness here when we just have I in the population and if you think about a slight deviation from that we can approximate it as being at this point here that's the same in both these cases but on the left hand side as soon as we have Jade trying to invade it has a lower Fitness so as soon as we get pushed away from this point here where p is equal to one then WJ is less than wi so we get pulled back towards it essentially you can't move away from it so this is an ES whereas over here as soon as you move a tiny bit away even though they have the same Fitness when p is to one J has higher Fitness so it can in okay so it is a Nash equilibrium because they have equal Fitness but it's not an ESS so this is Nots okay this is again a weak it's pretty easy to identify pure strategy dsss from a payoff matri because a pure strategy I is going to be an ESS if no other strategy can invade when R so the largest value in the I column of our payoff Matrix must therefore be the pay off against itself so think about this question below which strategy in the following Matrix is a pure strategy ESS and is this a strong or weak natur equilibrium well suppose we've got a matrix a here I'm going to do a 3X3 game suppose we've got - 1 0 3 0 1 4 2 - 2 and 2 okay so which strategy in this Matrix is a pure strategy ESS strategy one two or three stry 2 why no carry on no I mean you had the right answer so so that if we look in I column so first of all let's look in the The First Column so if we're looking about the the statement here is saying that when we look in the I column the entry on the main diagonal has to be the highest entry in that column okay so if we look in this column for example column one the i i entry minus one but two here is higher so this can't be an SS because there is a higher value in this column so strategy one cannot be an esss if we look at strategy three along the main diagonal two is the payoff here but there's a higher payoff if we were to switch to a different strategy and then if we look in the middle column the highest payoff is on the main diagonal so this is an ESS so it's a really nice simple way of just looking at a payoff Matrix to figure out whether we have an a pure strategy ESS this case strategy 2 pleas yes sure a I means that I andon mhm and value of it yes so if I say a should be greater than a means that if the opponent is going to change it strategy my payoff should not decrease yes so but we can look at it the other way around that if I change my U if I change my strategy M and the opponent doesn't then my increase again depends what your depends what your opponent's playing the is the opponent playing the same strategy as you to start with yes okay because way that individuals switch to different INE their so yes if I try to switch my straty I should compare a i with Aji as well this the meaning of this sentence comparing a i Aji uh yes so that's what we have up here a i is comparing with Aji yeah so a so that would be me changing my road so that's just like here if AI if is2 is here is this pay up one the opponent changes strategy and then and then my I think if your opponent changes but the I think this is aad thing it depends well it depends on the frequency of that opponent changing right because we're talking about ultimately like populations here so if the if your opponent changes strategy well if it's just one opponent changing then you can't like they're not going to have a big effect so the one way to thing about this that we'll get on to at some point later is that is ideas of local stability right so this is equivalent in some sense to local stability you're familiar with that right so you can think of there being a key issue in local stability let me sketch it on here if anyone's this is a slight side but when we're thinking about dynamical systems for example suppose you want to the equivalent here is you want to get to a fitness Peak okay so you might be this fit this peak here but there's a globally optimal one up here so this is a this is still an ESS when we come to this this is in SS this is also an ESS but you can't just jump from this one here to together okay you would have to sort of decrease your Fitness go through them so ideas about ess's are often also tied in with ideas about local stability okay so if you s like perturb the system greatly and everyone else shifts to the other strategy then yes you can switch to something that has a higher Fitness or your Fitness might also you could Shi from this one down to here right um so I see what you're saying but it this is all about um making essentially like slight changes rather than everyone wholesale shiting because this is not the this is not the biggest payoff but this is the biggest payoff um for an individual against itself which can't where where other strategies don't have a higher payoff right so you this is the highest uh payoff here but if everyone switches to strategy 3 then we have this issue of moving away from it right some of these Concepts around evolutionary stability are kind of like tricky to get your head around wait until we get on to things about evolutionary branching because then you're talking about walking through Landscapes that are changing as you walk through them um but yeah the main thing I guess to think about this with Nashi calibria is that the distinction between Nash equilibria and evolutionary stable strategies is Nash equilibria are almost like an evolutionary stable strategy but don't have this condition about Invasion so something rare can't invade okay so see what time so what if no strategy is better against I than itself but at least one other strategy is equally as good so this is our weak national equilibrium event this means that some elements in column I are the same as in column aii or sorry as the same as entry aii but none are greater so for example if we change that Matrix that we had before now we have something that looks like this so if we suppose that a II is equal to AI J sum J not in I to I so before our strategy 2 was our n equilibrium I've changed this entry up here now so this is greater than or equal to the other entries in the same column so a22 is greater than or equal to the other entries in the second column but is not strictly greater than them okay so we know that we can't be in a strict natural equal here could be in a weak actually cor so in this case what do we need to do we need to check whether I does best than J than J does against itself this is this Invasion criteria right so does this Matrix above contain a pure strategy ESS I've already answered is this is strong a weak n equilibrium it's a weak n equilibrium but maybe spend a minute chatting to your neighbor decide whether strategy 2 is a pure strategy ESS okay what do we think share hands who thinks that this isn't an esss who thinks it is an ESS and who's not sure okay let's go through this condition then so the first thing if we just go back to this Matrix here the first thing we can do is just look at any of our columns right and then we want to see is the entry on the diagonal in that column strictly greater than all the others if so it is a strong equilibrium in this case here if we look at our second column there's an entry that is equal to the one that's on the main diag there's nothing greater so it's going to be a weak Nash equilibrium and then if you're if we just go back up here a second that we have this additional condition to check in the case of a weak na equilibrium I refer to it as a strong na equilibrium usually refer to as a strict Nash equilibrium if we've got a weak Nash equilibrium then we can check this condition here so the entry on our diagonal so we're in column two and we want to compare the entry that is equal to the one that's in entry 22 so we have this additional condition here that we're checking we're checking that for all other entries in this column that our entry on the main diagonal is higher than that entry against itself which which is slightly confusing so what do we have well here entry a row one column 2 is equal to entry a22 we can also see that a22 is greater than or equal to a uh J2 for all J so this second condition here is for our n equilibrium and this is telling us that it's a weak n equilibrium now we need to check whether ajj is less than a IJ so check if a JJ is less than a i j in other words is j a worse response to itself than I is to J this is our Invasion criteria so this is saying that this strategy J while it might be as good as I it can't invade so we look at a i i so in this case sorry ajj A and J in this case is one it's minus one how does that compare well a21 is zero so just write this out a11 isal to minus1 so here our just put in Brackets here our J is equal to one here and our I is equal to two so if we look at a i j we're going to be looking at a21 which is equal to zero a21 therefore greater than a11 and so I = 2 is a pure strategy PSS we don't have to be a strict Nash equilibrium have SS we can have a weak Nash equilibrium like this all we need to do is check how that strategy that is as good as our best response performs when it's R that's just a wecking the on we are checking yes on there it's written as a j greater than Aji oh sorry that is that is a j but my writing is terrible so that is a j there should we should we check this for all only so this only matters for so J is equal to three but we only but we already know that this isn't an as equilibrium we already know that strategy 3 isn't an as equilibrium right definition SS for for every J um so you're saying we should be checking it for and J One J three [Music] sotis notf let see it is uh so if we switch uh if this if J is to three make sure I'm on the same page as you then our a33 is going to equal 2 but our a23 isal to four which is greater than two so yeah but you should check this one yeah okay okay so that's pure strategies what I'm going to do is tell you what let's have a break there I'll allow you to eat your tweets and then we'll come back and we'll talk about mix strategies and then we'll talk about some repeated games as well okay so have a break for say 10 minutes I'm back just after half past and we can carry on and for those who had their Suite stolen because I'm s you oh yeah oh if you could return your counters up here when you have a moment that'd be great thanks did you watch the playli and then you knew I've seen it way before years ago [Music] and this is our I this is supposed to be supposed to be J I might have got my eyes and J's yeah it's this part is yeah J it's part of the reason is is because when I do there's related sort of matrices in disease modeling right and it's round the other way yeah it's about it's the we talk about who infects matrices right and the rows and the columns are reversed typically so that was correct this uh that should be a see on this here okay I'll start up again hope person will come back um a couple of things so first of all you tell I don't usually do Evolution game theory because this is make mistakes there was a Tyle on this slide here than pointing that out this should have been a j i here an i j because we were saying a weak Nash equilibrium a little typo there and jaad just raised this question about this condition on this slide here on slide six really it's all about whether this has to be for true for all J or just the one that matches in the columns so whether whether we need this to always be satisfied with all of them or whether we need to have this only for AI J to a i i if there is some value of J that satisfies this we need to do this for all J it's a good question I'm going to go away and double check that and I will update the notes accordingly if it is different from this um it's something I'll have to sit down and think about so I don't need you think about getting so I will I will update the notes if that is it might be that either I've copied it down from the textbook incorrectly or the textbook is correct and I just need to get my head around it and make sure that I'm correct is I understand it properly and or it might be that the textbook is incorrect so I will update the notes accordingly and send out an announcement if it is different from what I've got written here okay okay so the second part today we're going to be thinking a little bit about mix strategies to start with and then we are going to go on to think about the repeated pronal dilemma so there'll be like the scenario that we started with um the beginning with the split and steel a little bit related to that then thinking about what happens if you're going to repeat that process and play the same person again okay so so we've just found pure strategies they're fairly straightforward to look at because we just need to look at the payoffs and the Matrix and then we can see okay well because it's a pure strategy then individuals are playing that strategy all the time there's no sort of uh or rather the probability distributions are trivial because individuals are just playing them all the time um mixed strategies we have um a non-trivial probability distribution which we can describe by a vector so a vector P could be a mixed strategy and suppose it looks like this p is equal to P1 P2 P3 if we have three possible strategies in our population and there'll be some probability distribution of playing those strategies so say 0.3 0.70 there's nothing special about those numbers they sum to one so this would correspond to play strategy one 30% of the time strategy 2 70% of the time and strategy three obviously there's an infinite number of different probability distributions that we come up with describing different mix strategies you recall as well from the last lecture that if there are only two types of individuals in the system so if there are only two strategies then this is in distinguishable from for example if we had let's call it Q is equal to 0.3 0.7 if there were only those two strategies this could be everyone plays strategy one 30% of the time and strategy 2 70% of the time or it could be 30% of the individuals always play strategy one and 70% of the individuals always play strategy 2 however as soon as we go into to more than two strategies that second interpretation no longer applies so it's easier to just think of these in general as being a probability distribution where everyone plays each everyone has the same probability distribution say and they play these strategies 30% of the time 70% of the time zero okay okay so in general we want to also be able to find mixed strategies as well as pure strategies but they are much harder to find one of the things that we can do is we can kind of eliminate p uh mixed strategies we can do that by thinking about something called the support of a strategy so the support of a strategy P which we call F of p is going to be the set of All Pure strategies that can be played in so we don't care at that point about their actual frequencies as long as or their probabilties as long as they're above zero so in the example above F of P is just going to equal the set one two so strategies one and two can be played but you don't play strategy three this then gives us a simple rule it states if p and Q are both 's so p and Q here are just vectors describing different probability distributions so if they're both ES is then we can't have the support of strategy P being contained in the support of strategy here that means that if we were to have mixed strategies then as in ESS then you wouldn't be able to have uh there wouldn't be able to be any overlap in terms of the strategies like contained within those mix stres so for example consider this P of Matrix 4 213 so first of all before we go on are there any Nash equilibria here y okay strategy one is an actually equilibrium why is strategy one an act equilibrium yeah it's also ANS right because it's greater than one here we also see that strategy 2 is also an ESS so we have pure strategies then for then p is equal to 1 Z and qal to 01 but we can write any pure strategy in the same notations we write our mix strategies it's just a trivial probability distribution yeah so in this case no mix strategy can be ANS in this two play game because any mix strategy is always going to have uh both of these strategies within them and therefore the supports are going to be the same so we can't have them being an esss no ESS is in a 2 by two game that's not going to be true in general if you we can have more strategies right okay let's go back to the prison dilemma so if you recall with our prisoners dilemma we had a payoff Matrix that looked generically something like this we call the entries ABCD and I'm just going to write in the columns here sorry in the rows and columns headers D's for defect and C for cooperate D is to defect C is equal to a cooporate cooporate will collaborate here this is slightly different to the game you play because we have these strict in qualities here whereas you had the situation when you did your split will steal the sweets if you uh if your opponent stole then you always got zero no matter what you do so two of these payoffs would have to be useful okay let's get you guys working this out are there any Nash equilibria in this game and any ess's have a go with that for a minute okay what do we think any Nash equilibri in this game okay why is defect an equilibria uh yeah so if we think about okay if we look at each of our columns then yeah in this column here a is greater than C and then if we think about this column here B is also than D so it would always be the best thing to do so D is a strong National equilibrium in fact and it is also an ESS since C cannot invade so although D is greater than a so the payoff for Cooperators against Cooperators is higher than defectors against defectors corporate can't invade because I think most of the time it's going to be coming into contact initially when it's rare with defectors and D is less than b so it's going to be losing out it's going to be less fit than the most common type in the population never it's not going to be a to invade even if it is the optimal strategy overall if everyone did it okay I want to try and bring us a little bit back into the Realms of biology a bit more and thinking about the relevance of some of these games to biological scenarios so I've talked a lot about ideas of cooperation and cheating sorry the ca's a little bit dodgy ideas of cooperating cheating which obviously Game Theory links in very tightly but I haven't talked too much about any specific examples hopefully that will stay in uh one uh sort of model system that people study often is uh dict steelium uh it's a genus of proest or otherwise known as slime molds okay and often here there'll be groups of sorry I don't know why this isly disconnecting all the time trying to keep that steady so these slime molds they're uh very easy to study um and they're a classic example of cooperation and cheating and so what happens is these cells they live independently but then uh when conditions uh maybe change in the environment maybe they run out of resources locally maybe something uh environmental stresses appear they uh form something called a fru body this is a Fring body here so they group together even though they're individuals into these clumps of 100,000 or so um genetically distinct individuals so they group together to form this structure and it rises up and then they have this fru body at the top so these cells have differentiated into these different cell types to make up the structure either part of this stalk or part of this Fring body and the Fring body enables foration and dispersal so these spells can disperse hopefully to areas that have more nutrients and things like that more resources and everything left on the freeding body is Left Behind to die so I think something like 20% of the cells maybe um Will Survive and 80% are sacrificing themselves so this is a problem in the sense of like you know why would individuals choose to be in this Fring body shouldn't they just like rely on others to build the stalk and then they end up in the Fring body and then they can pass on their genes and they're cheaters essentially they're not contributing to the public good of producing this breeding body we say if everyone does that then the whole thing's going to collapse and no one does anything um but evolution doesn't have that foresight and so one of the things we often see is that uh and this kind of links back to some of these ideas you talked about the other day of the hawk Dove game that Evolution doesn't do what is best for a population or best for a species does what is best for an individual it doesn't have any foresight it can't think about what's going to happen in the future so what we can find sometimes is uh we'll get on to ideas later in the course as well about evolutionary suicide where Evolution can drive populations extinct evolution is not always advantageous so the question then is like under what conditions do we expect cooperation to evolve sorry I do not know why this cable is suddenly decided to die today okay some reason the projector seems to have gone into a calibration mode it's okay I will carry on talking anyway does everyone have the notes in front of them just in case I can't get the screen back up okay I'm going to try and go through this material quickly of course it fails again okay the point is that there are some presents interesting problems about when do we expect cooperation to evolve game theory is one way of thinking about that process um I mentioned this idea of a repeated prisoners dilemma so one of the problems with the prisoners dilemma game and a lot of the games we've been thinking about so far is that we're trying to adapt these ideas from gain Theory which are about what happens if I come into contact and and have you know one resource what's going to be my expected outcome but it's not really accounting for the fact that uh individuals may adop strategies that depend on repeated interactions so you won't typically just interact with that one individual once or you encounter an individual with that strategy once we' expect individuals and the genes that they carry more importantly to encounter each other again and again and again and this could be within a single generation or it could be over many generations when we're thinking about the genes that they carry okay so we need to think about a much broader range of strategies and games where we have possibilities of things like retali or punishment so depending on what had happened in the past maybe you change your strategy so you don't necess adopt the string same strategy every single time so this is an example of uh this is this does happen in in biological populations as well so Guppies are are fish and one thing they can do is they can cooperate to inspect predators and uh experiments have shown that they punish those that don't participate in that sort of inspection of predators so there's a slight risk obviously of of of trying to inspect a predator you may be exposing yourself to predation so individuals that don't contribute to that inspection um they are um essentially punished by not being allowed to associate with the ones that do cooperate with each other so even fish can retaliate remember that next time you go fishing or you're having fish and chips if your Bri okay so what we want to do is we want to adapt our prisoners dilemma game to allow for needed interactions and one of the things we can do for these payoff matrices is if we just go back a moment up to here I just had generic values a c and d one thing we can do is just rescale this and rescaling it doesn't qualitatively change anything so what I'm going to redo is I'm going to rescale this and call this new pair of Matrix a hat again it's going to be for defect and cooperate in our rows and columns like this and all I'm going to do is I'm going to set my a value to be zero this a slight rescaling the only reason you do this is just because it helps the analysis a little bit later on so you still got the same conditions essentially as before that b is greater than D and D is greater than C and also here we're going to have C's got to be negative now like I said it doesn't change the Dynamics of the game but just helps to simplify the analysis a little bit what we're going to do now is we're going to think about a probability R the two players that who are currently interacting with each other so they're currently facing each other they will meet again in another round in the future with all rounds being independent of each other okay so this would be like you guys pairing up as we did at the start and then we do another round of that and you randomly choose another pair what's the probability that you're going to choose that person again well it would be one over the number of individuals in this room minus one can't CH okay doesn't matter what that probability is are how it's determined we're just going to assume that there is a probability they meet each other again and so we can think about the probability that they meet each other n times is just going to be r n this is going to be a simplified version as well in the sense where either these organisms live forever or we're just thinking about over a short enough period of time you suppose they all live for the same length of time how many times do they interact during their lifetimes okay so we can think about the expected number of meetings they're going to have between two individuals who encounter each other at least once is going to be what the expected number of meetings well they've counted each other once it's going to be one and then they encounter each other again a second time is going to be probability R and then again probability R 2 plus R cubed plus so on any want tell me what this is yeah so we can just write L is equal to I'll do this in a different color so it doesn't get confused L is equal to an infinite series the you've forgotten how we solve these one thing we can do is we can divide throughout by R we have one r + 1 + r + r 2 and so on because we're dealing with infinite Series this thing here is the same as this thing here so l r is = 1 R + L do a little bit of rearranging here we multiply throughout by R you have L is = 1 + LR subtract LR from both sides and we'll get L * by 1 - r R is going to equal 1 or L is = 1 over 1 - R is everyone okay with this St what I did here yeah good okay so we know now what is the the expected number of meetings that we're going to have for the two individuals who have met once the question is what are our strategies going to be and there's actually been quite a lot of research into this as to like what is the best strategy in repeated pris St game because it's kind of the archetypal game of cooperation lots of people have sort of tried to figure out okay what strategies can we play the problem now is that we have an infinite number of strategies that we can play infinite number of different strategies because we could uh you could base your strategy in the nth round on anything that happened in any of the N minus one rounds so we've got an infinite number of strategies because we're meeting essentially an infinite number of times two common strategies that I talked about are always defect which as you might expect is to just always play D so always defect when I say play D here it just means that that's the strategy that you employ regardless of what the other one does another one is tip for Tat and this is play cooperate first play C first in the first round and then copy what an opponent does opponent did on last encounter in the first round you cooperate and then after that you just remember what the opponent did in the last round and you just cop copy what they did in the last round there's another common strategy um remember what it's called now but in that one you don't have to remember what the what your opponent did you just have to remember what payoff you received itself and then depending on the payoff that you received then that informs whether you corporate or defect it's a slightly different version ENT there's been various competitions to try to find what is the best strategy um as a as a general principle in this infinite game okay so using this information above and from the previous slide what I want you to do is to construct payoff Matrix for the repeated prisoners dilemma so I want to have a matrix a which has always defect against tip for tap and I want to know what are the payoffs based on what we had on this slide here and this slide here in fact I'll write those things here so our original payoff Matrix for one interaction was z b c and d and our expected number of encounters is 1 1- R so have a go using that information to write down your payoff Matrix for the P prison di okay sh F he got that not sh yeah okay let's go through each of these to make sure everyone's on the same page okay so if you play always defect against always defect well in the first round your payoff is going to be zero in the second round your payoff is going to be zero and the third round your payoff is going to be zero and so on an infinite sum of zeros here always defect okay if you play always defect against tip attack what's your payoff going to be in the first round b y okay so you defected in the first round what happens in the second round what's your payoff going to be in the second round why is it going to be zero yep okay and then the third round okay we got zeros of infinity then okay so if you play always defect against playing ATT get B in the first round but then after that you get nothing what about if you do tip attack against always defect what do you get this time in the first round bottom left c yeah and then second round Y and then zeros very on and then lastly tip for tap against tip for T what do you get in the first round second round always do right okay so we can simplify this then as zero B C the question is how many D's do we have well our expected number of uh encounters is going to be l so D * L which I'll just write as D * by 1 1 - r r slightly unclear here the these sums here are based on what you would get per round this is not really a strict this is the payoff Matrix here and think of this is just sort of thinking what those infinite sums are we then have to multiply those infinite sums by how often you're going to expect to come into contact with said individual okay so this is our payoff Matrix when we have an infinite series the question then is when is tit forat in ESS is always defect in es and what about if we sketch a graph of the payoffs so I'll work through those so T tip for Tat is going to be an ESS if what let's have a look at this Matrix up here well we know for this to be an ESS that this entry here is going to have to be bigger than this entry here and that the other strategy is not going to be able to invade it right so that means that we need D over 1 minus r to be greater than B and B is also positive so this deals with the zero condition as well so we have a condition based on our probability of future encounters and our payoffs the original payoffs one encounter B and D to give us tip Fort being an ESS okay how about always defect is always defect an ESS and if so when that it is remember that c in this rescaled game is negative so always defect against itself is going to always be good okay so always def is always an es since C is strictly less than zero so it's a strict n equal over it's in SS okay about the third part third part we're going to be asked to sketch a graph of the payoffs for each strategy against the frequency of our always defectors and I'm going to call this frequency of our always defect strategy e so how do we work out the payoffs for each strategy well you think about what they're expecting to get based on their encounters with other individuals in the population so it's the expectation for for def well if we think about their interactions with other individuals who always defect they always just get zero for those interactions so they get zero their payoff Times by the frequency of for different factors in the population p and then we have 1 minus P the frequency of t for T players Times by what well we learned above that their payoff for interactions with these individuals is going to be B this is just B * by 1us B my players who play tip for tap we can do exactly the same thing going move these fits over here so for these guys when they encounter someone who plays uh bo defect then they get a payoff of C which remember C was negative so you have C * 5 p as that expected payoff interactions with individuals who play for Def and they get 1 minus P frequency of tip tack players Times by their pay off d 1us r we can do a little bit of rearranging to get P by cus d 1 - r plus d 1us r note that both of these are linear in p and they decrease with P so we can sketch a graph of the expected pay off as the frequency of always effect increases pay so let's look first of all at our always defect so our always defect their payoff is going to be B when no one else is in the population so if there's only sorry if there's if there's only tip for tap players in the population they'll get a payoff Fe they get that in the first round and then they get nothing else for any of the second rounds and their graph will look something like this and then we've got two possibilities here depending on this condition up here I'm going to assume that this is satisfied that D is d 1us r is greater than b so if we have d 1us r here greater than b and payoff for tip for T will look something like this my tip T this is my always the fact so in this particular case we see that tipat is an ESS you think about ESS is on the diagram like this as being here tip for T is in SS because it has the highest Fitness when it is um at a fre of 100% p a frequency of one here ATT is the best always effect has the highest Fitness highest P when it is the only strategy in the population nothing else can invade that's why we can see that both of these are 's this is like that diagram I showed earlier of there being two mountains next to each other one of these may be higher than the other but they're both ess's because they're very difficult to invade locally you can't escape them they're attracting that sense so yeah uh we see that by sketching this graph of our payoffs we can see that TFT is an ESS as long as this condition satisfied and is true for some value P star yeah I P less than that P star our TF T is going to be able to um continue spreading it's going to have the highest payoff obviously if our D over 1us R is less if it was down here for example then we would have something that looked more like this this was our d 1us r this is our other possibility if this is less than b then TF is just never able to invade here's not an ESS always defect is always the best strategy so what does this tell us then well in situations like these Guppies say because of their repeated interactions with each other they should be judging their actions if possible based on what other individuals do so they could sort of always just you not bother to inspect their um not bother to inspect uh Predators but if there are enough individuals that do inspect and they play this tip for tap strategy then that will be the best strategy to employ as well so there's an element of punishment there in terms of they would not not in terms of asso but they wouldn't contribute to the Future to helping those individuals okay so it's somewhat analogous to this okay let's about it today we finished a few minutes early that's fine um so we've explored these different strategies um in terms of pure strategies we've looked at mixed strategies and we've thought about iterated or repeated games and they can tell us something like say more informative about traits such as cooperation um and this idea of an evolutionary stable strategy or ESS is going to appear again um when we look at another approach to pH Evolution so I mentioned earlier adaptive Dynamics or evolutionary Invasion analysis that tells us something about um not only what are potential optim strategies but also how do we get to those optimal strategies and what happens when we maybe reach a strategy that might look like it's optimal from AF far when we get there it might not actually be as good as we think it is so we'll get onto that a little bit later in the course we'll be thinking about how we actually um have Invasion and replacement of strategies as opposed to just competing strategies um and then thinking about not exactly the the process of evolution we're just here at the moment with these gains one of the problems is that we're just kind of a very very simplified version of evolution where there are these strategies we're not worried about in the genetics at all but we're also not thinking about the process of invasion and replacement of one strategy compared to another we're just kind of jump into the end okay so we will deal with those um apply in a couple of weeks we might get to it just before the midterm um next week we are going to be looking at some asymmetric games which make things a little bit more interesting than these ones that we've been doing so far so everything so far has been symmetric in the sense that you've had the same set of strategies available to you um and uh and your opponents but that's not always going to be the case for example males and females might not be able to have the same strategies because they are biologically different to each other so we can think about asymmetric games next week and then we'll be starting to think about what happens about this process of evolution okay uh other than that have a good week uh homework two is now available um engine week