Thinking in Bets: Decision-Making Strategies with Annie Duke

[00:00:00][Music] great so we'll get started Annie Duke has devoted her life to the study of decision-making under pressure during her career as professional poker player she won over four million dollars in tournaments earned in World Series of Poker bracelet and is the only woman to have won the World Series of Poker Tournament of Champions and the NBC national heads-up Poker Championship retired from poker since 2012 she is now a corporate speaker and a consultant on decision strategy merging her poker expertise and her graduate level research in psychology at the University of Pennsylvania she has authored five books is a co-founder of how I decide org serves on the board of after-school all-stars and is a trustee of the Franklin Institute she's also wanted a televised championship in rock-paper-scissors I think I won like ten thousand mom before and she lives outside Philadelphia so let's welcome her to Google so Amy want to start with what inspired you oh that's a very broad question so you know I used to Joe the joke that I used to make was I I for most of my professional career I spent my life taking advantage of people who made decisions that weren't so good so maybe I should spend some of my life trying to help people make better decisions so that's kind of the cheeky answer but I I think that what actually really inspired me to write the book was that I started off my life in academics I was I was at the University of Pennsylvania I was studying cognitive science which is really of course a study of the way that our brains interact with the environment how do we process information you know as it comes in I was getting my PhD in that you know had a National Science Foundation fellowship and I was on my way to become a professor and right at the end of that journey I actually got sick I ended up with a stomach problem that landed me in the hospital for a couple of weeks and I ended up missing the the job market for that year because it's seasonal so I I had to take time off just to recuperate and I was going to go back out on the market the next year and I I ended up in that year that's when I took this what felt like this very big left turn into poker so I started playing poker and about eight years into it I got asked by actually a retreat of options traders if I could come speak to them about the way that poker might inform their decision-making and how they handle risk and it was this moment where I realized you know oh gosh I thought I took this really huge left turn out of academics but I didn't I landed in just a really amazing natural laboratory to study how do we make decisions under conditions of uncertainty when the outcomes of those decisions are not very well connected to the quality of the decision-making itself it's just a very noisy environment and how are you pulling the signal out and how are you learning under those circumstances and the answer is not so well really and it was this moment of kind of revelation for me as I was as I was thinking like what am I gonna say to this group of people because I it was this kind of moment where it all came together for me where I said you know it's really interesting because I was taught in graduate school that learning occurs under conditions where you have lots and lots of feedback tied closely in time to decisions and actions and as I'm gonna go talk to these people what I realize is poker is the ultimate lots and lots of feedback tied closely in time I mean like within 30 seconds of time to decisions and actions and yet what I saw was that learning was not occurring very well and so it was that moment of wanting to square those two things together and after that first talk I just went along you know I was like oh and I started building this second career and and speaking to audiences about how pokers a kind of a really great platform and framework for kind of understanding not only what are the problems in decision-making but how can we kind of solve for them and I sort of feel like from 2002 I was just workshopping this material and that I was finding this incredible joy in sharing these ideas with people in a way that was felt really different to me than poker and poker I'm thinking about these ideas in a way that's very zero-sum right I'm gonna win you're gonna lose you're gonna win I'm gonna lose and you know when I started doing this in 2002 I started growing that business and it started to become a much bigger part of my life actually than the poker wasn't it it just felt in this really nice way like I was adding water sort of back into the ocean and you know that's this book sort of you know came out of that whole journey that I took so can we talk about what prevents people from making good decisions okay because that's that do we want to narrow that down at all what prevents people from making good decisions I think it's a collision of a couple of problems I think that the the first pieces is uncertainty and uncertainty comes from two places it comes from hidden information so this is where we can see how poker might be a really good way to look at decision making the first place it comes from is hidden information so in poker the cards are face down guess what in life as well there's very often a lot of information asymmetry right and there's lots of stuff that we can't know and there's lots of things that we don't know so that's the first place that uncertainty comes from but even if we did have perfect information say for example about a coin and we know that heads or tails it's gonna flip 50% of the time we also have the intervention of luck which is the other form of uncertainty so even if I know everything I need to know about the coin so there's no hidden information about the coin that doesn't mean I can predict whether it will flip heads or tails on the next try I can say something over 10,000 tries I can tell you how it's going to work out over time but most of what we're doing is actually sort of these one trial scenarios and particularly as we're trying to make decisions and learn from the outcomes of our decisions we're not very good aggregate or so we're not sitting back well first of all most of the decisions we make we couldn't get 10,000 trials even so but even if we could the data is kind of coming in one at a time as we're out sort of in the field making her own decisions and so we're at were sort of encoding everything one at a time we're not sitting back and saying let me not making any decisions until I have lots and lots of data and in fact that would be impractical like you for example you would never say get married to somebody that write because you'd have to get so many trials in order to know that you were sure so that it's not practical to do anyway so the first piece the first problem is that there's all this uncertainty and then the second problem is that nature the way that our brains has are built has given us a lot of rope to sort of hang ourselves with the uncertainty and we can sort of go back to sort of the beginning which is what are the types of errors that evolution built us to make because so there are type 1 errors and type 2 errors type 1 errors are false positives and type 2 errors are false negatives and we're really really built for false positives we connect things together we think they're causal we think they go together much more often than they then they actually do so we're big at patterns we love things to be connected together it's a and that causes a problem so that's sort of problem number one and we can think about that in the sense of which is worse if you're on the savanna and you hear rustling you know among the reeds are you is it are you gonna wait and say oh let me try to figure out if there's actually a lion here or you're gonna run away right so obviously we're gonna run away so that's number one number two is were built for tribes so there was a lot of advantage for us surviving if we came together in groups and we had tribes and so that causes a problem as well for decision-making because it causes these problems in the way that we process information from things that are outside of our tribe so that's kind of the number two problem and then the number three problem is is really I talk about this a lot in the book is that we aren't really built to disconfirm the things that we already believe we're very much built to affirm our priors and so when you think about the uncertainty and the way that we sort of interpret information as it comes in there's so much uncertainty in in the data that we're getting that it gives us the room to just process the world in a way to make us feel right about the things that we already believe and when you have that collision between that the false positive problem and then the tribal problem on top of that like things really fall apart now your book you're talking about that's beliefs people have about showing the right information to people and somehow I will get them to see things correctly can you talk more about that - sure so I think that this is part of the problem is that we have these ways that we process information so I hear an example that I talk about in my book is something called motivated reasoning so I think you've all probably heard about confirmation bias so motivated reasoning is a larger problem than confirmation bias so in confirmation bias we have some sort of prior belief and well notice stuff that confirms the belief and we just kind of won't notice things that disconfirm the belief motivated reasoning is let's think about this as an umbrella problem of which confirmation bias is part which is that we actually are motivated to confirm the beliefs we have and that drives our information processing so that means that not only will we process them from well we sort of notice stuff that confirms us and not notice things that disconfirm us but also when confronted with information that disconfirms as we work very very hard to discredit it and when confronted with information that confirms us we actually don't put any effort into the discrediting at all so we don't really vet it very hard so I and you guys can probably feel that right when I read a article that say politically agrees with me I'm like wow that's a really smart article and when it disagrees with me political I am like that you know they're looking at the study and the N is too small and they weren't looking at this data and they were asking the wrong questions and the person isn't reliable anyway and they forgot all this other stuff and it they're totally biased and you know it's weird because I just don't go through that process with the things that agree with me strange okay so so not only that then we have this other issue which is that when we're in a tribe part of the part of what a tribe gives you is kind of fourth it gives you a sense of belongingness a sense of distinctiveness right so you belong to a tribe you're distinct from other tribes in some way where you process that as being superior it gives you the idea of sort of what's moral and what's not is also including the tribe but then it also gives you something really important for this talk today which is called Epis closure meaning that the tribe is telling you what knowledge is true and trustworthy and what is not so I think that a lot of times what we sort of think and this is where we understand this the motivated reasoning with the collision of the tribal problem what we sort of think is oh if other people just had the information we'd all be fine because then we'd all sort of believe the same things right so if we were all working with the exact same information we'd all be good but that's assuming that we live in something that we're all living in epistemic bubbles versus echo chambers so let me just explain what the difference is an epistemic bubble is you have you have information and you don't have other information period so if you're in an epistemic bubble where you just haven't been you just haven't gotten the information then if you get exposed to the information you incorporate it and you're all good but epistemic bubbles are pretty rare what we mostly live in is echo chambers so echo chambers are it doesn't matter whether we have the information or not people outside the group are not trusted sources so if you get information that disagrees with you that's coming from someone outside the group it's not trusted anyway and you can see how you can use that in order to accomplish the goals of motivated reasoning so when you're now confronted with information that disagrees with you the way that you actually one of the ways you discredit it is to talk about the trustworthiness of the source so you know what we want to be really careful about is we want to try to apply as we're trying to imagine and I talked about this in the book is we're trying to sort of vet information that comes in we want to apply a particular norm called universalism which is separating the person who's delivering the news from whether the news itself is correct or not so the way I put it in the book is it doesn't really matter whether it's you know Copernicus or Mussolini or your mom who tells you that the earth is round the the fact of the roundness of the earth is independent of those sources so you know in order to actually be good Universalist it's usually pretty good as you're sort of having these reactions to information to try to imagine absent of the source or to try to imagine it being delivered from a source that you actually trust and and that actually helps you to vet the information independently of the source which were really really really bad at and it's part of this problem of echo chambers so you're talking about systems of how we can overcome that and looking at decisions in terms of bets you sure yeah so you know the practice of the book is called thinking in bets and and the premise is that if we were more explicit in imagining or in in actually just making clear that every decision that we make is a bet that we would actually better be better off and it would create a lot more open-mindedness and why is it well first of all let me try to convince you guys that every decision is about so that's the the first step so I actually I'll tell a story that I tell in here which is kind of a fun story I think so there was a guy named John Hennigan a really really really great poker player known to have a lot of gamble and in the 90s when he was in his 20s he was really if you were to think about sort of what is this if you imagine the stereotype of a poker player in in 1995 like John Hannigan would be the guy like he was you know he never saw the Sun he was also he playing at night he was like a total action junkie like when he was finished at the poker table he just wanted to gamble more like he was really he was very good but he really liked a lot of action so he was sitting at a poker table one day and a discussion broke out out about state capitals and people were like quizzing each other on state capitals so des moines comes up and John Hennigan actually announces that the table is they're discussing des moines oh I could totally live in Des Moines and you know Des Moines obviously had reputation is kind of sleepy maybe if you were to think about sort of what's the antithesis of Las Vegas maybe you would think of Des Moines it's an example so the players at the table were like oh come on like there's no way you could live in Des Moines but Hennigan actually at that time in his life was sort of thinking like ah you know I never see the Sun and so maybe it will be really nice to go and sort of like detox from Vegas a little bit and then he also another thing in mind which is that poker players gamble on golf a lot and he thought well then I could actually really spend some time on golf without the distraction of the actual poker so they negotiate a bet so at the table they negotiate a bet that John Hennigan is gonna move to Las Vegas for 30 days I mean to Des Moines rather for 30 days they add a little bit of a diabolical condition which is the hotel that he's on is only one Street and there's only one restaurant on the street and there's like no bars there's no nightlife there's nothing he could he's confined to the one Street when he moves there for the 30 days except there's a specific golf course he's allowed to go to the practice golf during the day so they settle on thirty days and the bet is set for thirty thousand dollars so if he can stay in Las Vegas for thirty days he wins thirty thousand dollars if he can't stay there any and he has to come back from Des Moines then he's gonna lose thirty thousand dollars so literally the next day he moves to Des Moines so I two days later he calls up the group and he says hey I'm in Des Moines I'm obviously like I've completely up and moved to Des Moines clearly I'm good to go I'm gonna stay here so if you'd like to settle you guys can pay me fifteen thousand dollars and I'll come home and I'll save you the misery of losing the whole thirty yeah that was the group's reaction so they were like no we're good John why don't you you know why don't you stay there so a week later John Hennigan had moved back from Des Moines he was he was living in Las Vegas and he had paid the group fifteen thousand dollars for the privilege so um so I tell this story because it sounds like oh my gosh these crazy poker players making this bet on where John was gonna move but if you think about it it's really no different than any decision that you've ever made I mean your decision to come and work at Google right you have other opportunities you're thinking about what John has to think about am I gonna like Des Moines he doesn't know cuz he hasn't lived there before right the money that I'm gonna make if I win this bet is it going to be better than the money that I'm gonna make if I stay in Las Vegas and if I'm actually taking less money is that okay because I'm creating different types of opportunities for myself like I'm gonna feel like I live a healthier life for that month and that that's worth making less money or I'm gonna work on my golf game and that's gonna have payoffs which would be sort of the same as I'm thinking about taking a job in Des Moines and I have to think about how it's Des Moines gonna compare to the place that I live now I don't know I've never lived there am I gonna like it versus any other place that I might possibly move is the salary good enough for me to feel like I'm willing to take that risk in terms of going and moving there what are the opportunities for advancement what other kinds of money that you know could I make in the future and when you choose to move to Des Moines or relocate or take the job at Google you're obviously foregoing all other possibilities available to you you can only make the one decision which was true for John Hennigan as well so basically Hennigan had a certain set of beliefs about whether or not he would like Des Moines whether or not it would be worth it to him and when he made the choice he was foregoing all the choices including staying in Las Vegas and staying with the status quo we're moving any other place where he might maybe detox or go you know detox from the nightlife or go and you know practice his golf or whatever it might be and there was a whole set of possible futures that could occur from that one where he stayed in Des Moines for 30 days and loved it one where he loved Des Moines so much he never moved back one where he met the love of his life in Des Moines you know one where he calls up after two days and says hey I want to settle and the thing is that you don't know you're making you're making an estimation of what the probability of each of those futures might be and you're deciding that whatever you have to invest and it doesn't have to be money it can be your health your happiness your time whatever it might be usually it's all of the above whatever you're investing in the decision that you you actually go with that that's going to hurtle you to the best set of possible futures as compared to any other decision that you might make so the poker players really just make that explicit and that's really what you're doing when when you're betting is you're making it explicit that a decision is a bet informed by your beliefs on a set of you know possible futures that you might be hurtling yourself to so once we kind of understand that frame I think that it actually can help us overcome some of these problems that we have with decision-making and I'll show you a really simple example so here's something that's been in the news a lot lately because I hear it every single day on the news the Democrats are gonna take the house and default so okay so let's say that you said to me the Democrats are gonna win the house in the fall and let's say that my response was do you want to bet on that so think for yourselves what does that do to you it causes you to retreat from a place of certainty into a place of probabilistic thinking right because now what you start to think is okay well wait why do I think that there's a lot of time between now and November there's a lot of things that could happen right like Trump could solve North Korea I don't know that might change things right we don't know you know there's all sorts of other things that might happen with Trump Trump is somewhat unpredictable there are things that could happen with the Democrats there's you know I mean the number of things that can intervene on the luck side between now and six months from now is pretty large right and so so you'll start to think about that it brings it bubbles that uncertainty up to the fore and then you also start to think like what well have I been reading opinions that disagree with that like what what is the person who's challenging me to the BET know or think that I haven't thought of or that I don't know I have to start thinking about why might I be wrong here before I'm willing to actually put that kind of skin into the game right and so essentially what this element of let me think about my decisions explicitly is as Betts does is it forces you into more probabilistic thinking into more the recognize the recognizing of these two forms of uncertainty that I don't ever have perfect information for pretty much anything right we just don't and that the future is always uncertain that no matter how well I think about the future how much I predict the future even if the only bad outcome can occur 1% of the time it can still happen so I need to think about that as I'm sort of announcing these things with certainty once you start to bubble up the uncertainty in any decision that you make it causes you to be much more open-minded because now what happens is that you're thinking about well if I can only think about if my beliefs are probabilistic and my predictions about the future are probabilistic then I need to be information hungry because what I need to do is start refining my beliefs and refining my predictions because I actually have to bet on these things with things like my time and things like my money and things like my health and and these limited resources that I have and if I can only make one decision at a time I better stand I'm thinking about this probabilistically I better start really working to try to refine all of this stuff and how am I gonna do that well it's gonna cause me to be open minded in particular to why I'm wrong open-minded to opinions and beliefs that disagree with me because I already know why I'm right that's how I got to a place of certainty in the first place so once I start backing off of that I've got to start thinking about you know again what does the other person know that I don't know and I think that that's why it's such a good frame for trying to solve for some of these problems so you make the bet and it doesn't work out for you like what what should your next steps be so you make the bet and it doesn't work out for you right and so what should your next steps be so that's a very good question so it's actually it's actually one of the big problems is that once you have a result it really distorts your ability to go back in and analyze the decision process it's a problem called resulting so another story from the book that I'll give to sort of illustrate this problem and then we can talk about solutions for this is so do you guys remember the 2015 Super Bowl Seahawks vs. Patriots yeah so you're in the right area to remember that right so I mean so okay so it's the last play of the Super Bowl the Seahawks are on the one-yard line 26 seconds left it's second down they have one timeout and they're down by four so this is obviously a really important decision that's going on here because if they score the Patriots are unlikely to have enough time to be able to march back down the field and score again so if they can get a touchdown here this probably given be the game winner which would be a really big deal because the Patriots like never lose a Super Bowl until they fear and I'm from Philly but just sayin I was pretty excited about that so okay so what does Pete Carril do here he's got a running back on his team named Marshawn Lynch amazing short yardage running back everybody's expecting him to hand it off to Marshawn Lynch and instead he calls pass play Russell Wilson passes the ball and the ball is intercepted pretty famously and boom done Malcolm Butler intercepts the ball the game is over the in game calling from Chris Collingsworth is pretty brutal on this decision you know Peak Herald that I can't even believe this call is such a horrible call and the next day I have to say that most of the pundits did not disagree there seem to be an argument among the major outlets about whether it was the worst call in Super Bowl history which was most of them but USA Today actually said it was the worst call in football history period so that seems to be where the disagreement lay so this is a really good example of resulting so we know what the result was horrible right I mean the result was one of the worst results in Super Bowl history we can agree with on that but that does not mean that it was one of the worst decisions in Super Bowl history and I think to just declare that that Pete Carroll hadn't put any thought into it I mean this is Pete Carroll he's in the Super Bowl for a reason you know I thought I thought was pretty spectacular so like I'll just ask you guys to do the thought experiment so we can see what the problem is what do you do once you have the result imagine that Pete Carroll passes that ball and the ball is caught for a touchdown in the endzone just take a moment to think about that now how does that decision feel now brilliant but how it turns out on that one outcome shouldn't actually affect the way that we think about the decision right I mean that's like saying if I flip a coin and you call heads and it lands heads that you that was a brilliant decision but if it lands tails that was a really dumb decision like it one outcome tells us nothing so so I but you can feel it you can feel that like the result acts like a gravity well and you can't climb out of it to look at whether the decision process was good or not it's really hard now just I just to not leave you with a cliffhanger Justin you know the probability an interception in that spot depends on how many seasons you aggregate across but it's probably somewhere hovering around one or two percent right so here they declared this like the worst decision ever and like a one percenter hit basically so I don't think that should really say very much about whether the quality of that decision was any good also by the way for anybody under who understands options theory remember they only had one timeout so if a very usual event I think it's about 40% of the time that the ball just is incomplete there what happens the ball stops and guess what they have time for two running plays so if they hand the ball off to Marshawn Lynch they're gonna run so much timeout that they're only gonna be able to get two plays off even with being able to call the timeout so they're only gonna get to two plays if they pass the ball and it's dropped they actually get three plays including those two running plays that everybody wanted them to run in the first place so it's a somewhat free option it cost you about you know this interception rate of about one percent for a more detailed analysis Benjamin Morris on 538 it has a great one highly recommend you go read it but so this is where we can see like that once we know the outcome once we know whether it's good or bad it's incredibly hard for us not to then retrofit the decision quality to the outcome so the answer is unless you've got unless you're running a Monte Carlo which we can't for most things what do we do once an outcome has occurred in terms of how do we move forward it's like ignore the outcome as much as possible it's actually the weird answer to that and I understand it's counterintuitive but it's kind of the only way to move forward is like okay it was one outcome I got to think about the decision in absence of that and we can if you want we can talk about strategies for doing that but that'd be great because you mentioned phil ivey yeah inside of looking at every situation look at the skill aspect of it what can I control what can I improve and then you have Hellmuth who I guess this quote is I'm would be the best poker player of all time it wasn't for luck yes so maybe I can talk about that spectrum in the stress sure so there's only two things that determine the way our lives turn out luck and the quality of our decisions so we can't really control the luck element so we should try to get down and focus on the quality of the decision so you mentioned Phil Ivey I think it's a really good example part of the way to do this so he I tell a story about he won a really huge tournament against really an all-star final table Phil Ivey is one of the best poker players in history and he actually went to dinner with my brother and he spent the whole time talking about all the mistakes that he had made so obviously he's got an interesting cognitive style probably one that's partly responsible for for how good he is but he gives us actually a clue of how to how to handle these kinds of situations so there's a variety of different what we're trying to do is get to a point where we can all be Phil Ivey it's very hard to do it's very hard to sort of take our great outcomes and look for mistakes it's very hard to take our bad outcomes actually and look for mistakes as well we kind of want to pawn that off on to LA so what I really talk about in the book is that the way into this is actually to try to separate outcomes from decision quality as much as possible so how can we do that there's a variety of ways we can do that let's talk what sort of strategy number one all of this has done much better in a group by the way so and part of that reason is because of this kind of gravity well that we end up down in this is the way our brains work it's very hard for us to parse this apart but other people can say our bias much better than we can see it in ourselves I mean you you can spot bias thinking a mile away right so like get some people in a group with you who will spot your bias and you'll spot their bias and then here are some things that you can do with that group thing number one is when possible deconstruct decisions before outcomes have occurred so that's really really good to do and memorialize it so now as you're thinking about the decision and what the outcomes might be you can write down what you think all the possible outcomes are so we can think about this in terms of the Pete Carril example and there's like okay maybe the ball will be caught maybe the ball will be dropped maybe will be intercepted you try to assign some sort of probability to each of those possible scenarios you think about what the extra things you're getting out of it there's some what are the payoff in that case I'm getting two running plays anyway so now what happens is that when the ball gets intercepted there people you know you're not it overreacting to it because you've actually thought about this in advance and you can actually point and say well there's the interception scenario we thought about that when we thought about the decision and it happens one percent of the time so let's not overreact to that and let go now change our decisions the way that we decide in the future to not call that play again because it's already up there so like a trial lawyer can deconstruct their trial strategy before they get you know a verdict so this so that's like one thing you can do when possible do as much of the work before you get the outcome that's really good now in poker that's actually like literally impossible because you get the outcome in like 30 seconds so there's not a lot of work you can do in that 30 seconds and you can't certainly can't go off to a group and say can we work out the decision tree and then I'll come back to the table and actually make my decision so you can't do that so in that case what you can do is when you go and you work with whatever your true seeking group is that you've sort of formed together the people that you deconstruct decisions with only describe the hand or the question or the decision up to the point that you need help and know no further so like an example I could give from poker would be okay I've got a hand I you know let's say I had like you know I have King Queen and I opened the pot for a raise and this other guy rear aces and he's very aggressive and he actually raises a lot and so I was trying to decide you know should I fold or maybe I should reread him what do you think so notice I don't I know how the hand turned out I know how the person responded I know I would give a lot more detail than that leaving up to that but I know exactly what happened but if I'm asking you I'm not telling you because our our own bias these problems are infectious so once I tell you the way it turned out like let's say that I told you so I decided to re-raise and you then moved all-in on me you're gonna now make a story make sense and you're gonna now tell me about the decision in a way that's gonna make the fact that that particular result happened makes sense because that's just the way our minds like to work so I'm just gonna leave it out and I'm gonna only gonna tell you to the point that I have a decision so that's one thing I can do another thing I can do is to one person I can tell you it worked out well and to another person I can tell them it worked out poorly so I can change the stories and I think that's a really interesting exercise to do because you will be amazed at how different the analyses you get what the what the advice looks like from the person you tell that it ended well versus the person that you tell it ended poorly and I think that then you can sort of look at those two things and kind of work to merge them together in order to get some fidelity and your ability to actually understand what the decision quality is so that's like another way that you can sort of not infect people and you can do this in a variety of ways not just with outcomes but with your own beliefs so like here's a really simple so there's a simple problem which is like if you allow people to talk to each other they sort of try to come to consensus and make beliefs make sense and you're infecting everybody with your beliefs so like here's an example let's say that you have for people who have interviewed somebody that you're thinking about hiring don't let them talk to each other make them give you their opinions write down their opinion separately because once they talk to each other they there's all this that you literally infected it's like a virus another thing is like let's say that you're now in front of everybody and you've gotten the advice from people and now you want to start talking in the room about the advice do not say who you think should be hired because that'll ruin the whole thing right so like keep keep your own beliefs to yourself when you're trying to get advice from other people and this is a simple like in everyday life if I read an article on politics when if I'm asking you what your opinion of the article is I should not say what my belief about that is because then I've ruined the whole thing so I think that that's really what you should do with it it's kind of like unless you've got a large enough end which is rare you should kind of leave it out where do where two outcomes help you well you know obviously throughout your life as you kind of get them in they might be prompts for you to go in and say well maybe I should look at the decision quality but it should be both in the case of whether you won or lost it should be a prompt to go in and look at the decision quality and when you go in and look at the decision quality and you go want to talk to other people about it you're gonna do better because you've been prompted by the outcome to leave the outcome out as you're going to actually improve the decision quality it's a little counter into but there's some situations we're just sitting out not making this as you make sense because you mentioned that most poker players like 80% of the hands they don't make any actions on there's like 20th they play yeah so um so in poker one of the biggest mistakes that people make is actually playing way too many hands and you always need to think about when you're actually gonna enter into something you need to be selective so I want to talk about sort of both sides of this puzzle one is that just doing stuff isn't enough like I may think a lot of times people are like just throw a lot of spaghetti against the wall but you should actually think about each strand and make sure you're getting a payoff for the Strand so the fact is yes if you throw a bunch of spaghetti against the wall some of that will pay off and some of it won't but like and I think that people look at that and they look at like for example VCS and they think what that's what their strategy is that they're just sort of like playing every hand and then some of them pay off and you know when they pay off they're really happy but that's not true they're there they are spreading their bets across a large basket but they're actually vetting each of the bets that they're making to make sure that they given what they know they think that the payoff is going to be big enough so that's similar to what a poker player does so that's number one but then number two that I think is really important to think about is that don't take that to the point of thinking that somehow making a decision is substantively different from not making a decision because they're both decisions so you have whatever the status quo is and when you're going along and just kind of going with the status quo trust me you're making the decision every single time you do that so I think that when we're trying to sort of think about doing these scenario plans and imagining what the future might look like we only think about it in terms of decisions that we think are new we're somehow we're changing course but not changing course is a decision in and of itself so you should be taking time frequently to look at what the status quo is to look at what you're doing the strategies that you've always just kind of been applying and do the exact same kind of analysis on that as if it were a new decision so let me give you like a super simple example it's like the simplest example ever you're in a grocery store and there's like you know six lanes open and you're Einstein at this moment like you're literally doing some kind of quantum physics analysis about which line you're supposed to get into you're laughing because you know it's true so it's like you know you're looking for the lady with the coupons and you know the cashier that looks maybe like you know oh he seems a little slow or he's new or he's a trainee or you know the person who has 17,000 things in their cart versus you know whatever I mean you're it's like literally okay you are completely calculating this stuff out okay so now so because that's a new decision right so you're like calculating out all the scenarios trying to figure out like which is gonna give me the best future because I'm you're now investing your time right so this is like this is very clearly a bet you're betting on a line and you think that the you that results from choosing whatever line it is that you go to it's gonna save time over the you that results from whatever other line you might choose so that's like your and that's what most bets are you're not betting against other people you're betting against other versions of yourself right like Annie that chooses line I a versus Annie that chooses line B or like I'm in the restaurant Annie that chooses the chicken or Annie that chooses the fish or whatever so Annie that chooses Des Moines forces Annie that chooses Las Vegas okay so now you choose a line and you get in the line and the line turns out to be slow what do you do not you stay in the line you don't move but it's a new decision like once you figure out the line is slow switch but you don't because you know why because you're worried about like now if I switch what if that line my line starts moving faster I'll feel like I made a mistake but at any moment it's the same decision so if you treat the status quo as if like that's not a decision in and of itself what happens is you actually end up not changing course enough so this actually we can go back to the Pete Carril example to think about this let's do the other thought experiment where Pete Carril just doesn't change its grocery line right so he hands it off to Marshawn Lynch right that was the expected play that's the status quo play and let's say he doesn't score what do you think the headlines look like the next day I New England's good like literally nobody's like oh he's the worst coach in Super Bowl history because he handed off to Marshawn Lynch and he didn't score why because people aren't viewing that as a decision right like that's just the status quo we already have consensus around that clearly if he doesn't win then it's not the decision quality now we can firmly stick it in the luck bucket because nobody's questioning the status quo so now I must be something that's outside of his control so included and luck would be some for example the quality of the defensive line of New England would be included and luck luck is anything you don't that you yourself don't control okay so and that's because like alright he chose his grocery line like he's not switching so the reason why Pete Carroll got so demolished in that decision is cuz he switched grocery lines but he was right to do that like that's how progress gets made mm-hmm and I find their daily life to sometimes after result happens we'll look at the person who made a decision and then will say there are stuff well if it went their way then odds are it's luck but if it would have failed it's like you know it's mostly due to their like in competencies or whatnot do we is there mechanisms where we can be more like cognizant of that when we fall into that mode yeah so all right so I've talked about this particular thing resulting which is where we like really tightly connect outcomes and decisions so but there's two ways that we do this one is like we're looking at someone from afar like Pete Carril okay so he's not in our reference group right like we don't compare ourselves to him and so in that case we just look like bad outcome means bad decision and good outcome means good decision so it's interesting because we do a pattern for ourselves it's a little bit different so if anybody's read like Thinking Fast and Slow from Daniel Kahneman you'll remember from this from this that we're really trying to drive a positive narrative or of our lives right we want to think that we're good actors and we're smart and so and so forth so obviously this goes in the motivated reasoning category like if we have a bad outcome for ourselves if I lose a hand at poker that doesn't really drive with this positive self narrative at least not in the moment okay so then what do we do now remember we've got this noise in the system right hidden information and luck when I have a bad when I lose a hand at poker it's because of luck when I win a hand at poker it's because I'm a genius that's called self-serving bias I feel that feel like it's pretty self-explanatory named for that particular bias and we do this in all areas of our lives when someone gets a promotion you know it's not it's bad luck for us like it's not like we didn't do the work or we didn't you know obviously we deserved it it was just like bad luck it wasn't in our control actually car car accidents are an interesting place to look over 90% of two car accidents are reported as the other person's fault this is a pretty good one I think it's 37 percent I think of single car accidents as reported as not the person's fault I don't know you go figure that one out so that's how strong this bias is right now what's interesting is but the number of decisions that we make in our lives is obviously much less than the number of decisions were able to observe in other people so let's assume that we're really we're crappy for ourselves but it doesn't matter because then we get to feel good about ourselves but we're really good at analyzing everybody else's decisions but the problem is as soon as someone's in our reference group as soon as some it's someone that we feel like we're comparing ourselves to and sort of figure out what our self-worth is because we don't do that our happiness doesn't derive in absence of the way that we compare to other people now what we do is the reverse pattern right when they do something good it's because they got lucky and when they do something bad it's because they're a really bad decision maker so when they you know win a hand of Poker it's because they got lucky and when they lose a hand of Poker it's because they're a bad decision maker now you can kind of see actually if you think about Poker which is zero-sum how these two problems might have emerged right because looking at a zero-sum game like poker you can see how that pattern happens so if I'm trying to offload my bad results to luck and I lose a hand specifically to you I can't say that you won the hand because of skill that doesn't work so I have to say that you lost the hand because of luck because I'm trying to put my bat you know I'm trying to say it's bad luck for me so therefore it must be good luck for you and if I win a hand because of lot of skill rather I can't say that you lost because of luck it doesn't jive it's just that you know so we sort of treat everything like it's zero-sum like when we shouldn't but you can see how that happens when I specifically think I'm referring to you when you're in my reference group right so again like how do we how do we solve for this well I'm gonna go back to that group idea for I'm a really really good decision group that has different values so there's different ways that we can derive good feelings from ourselves right so our natural tendency is to try to offload it in the moment so that we don't have to have a negative update to our self-image and we don't ever miss out on a positive update to our self-image right because if I say that I won because I got lucky I don't get that positive update and if I say that I lost because of something I did I don't get that positive update and that's it or the natural way that we're like getting our pellets like we're pressing a lever in a skinner box is that that's our pellets are coming from but very early in my poker career I went up to somebody named Eric Seidel who's an amazing poker player he's won over 38 million dollars in his career so he's really really good I had met him when I was actually 16 and I didn't start playing poker professionally till I was 26 so he was just like a buddy of mine and then all of a sudden I'm 26 and now he's a professional peer so I remember I was in a tournament I got knocked out you know and obviously something unlucky happened to me clearly and I went up to Eric Seidel and I went to go what's called moaning I went to go complain to him about I can't believe this I had such bad luck I had this hand it would name a man the normal and he actually laid this out for me how do we solve for this and like the most beautiful way it's gonna sound kind of brutal but I'm so thankful for it so why are you telling me this like you think I'm not losing myself like why are you off loading your negative emotions onto me I don't need that if you really lost because of bad luck there's nothing to be learned from it so I would like you to not offload your emotional distress onto me thank you very much however if you have a question if there's something that you're trying to ask me about strategy I'm all ears so let's think about what he did there he said if you're not focusing on accuracy I don't care what does that mean this focus on accuracy let's think about the division but division division between the distinction between reasoning to be accurate and reasoning to be right reasoning to be right is I just want my priors to be right so I'm just gonna reason to prove that I'm right all the time right reasoning to be accurate that's funny I said right right after that reasoning to be accurate is I'm trying to construct the most accurate model of the objective truth and if you think about who's gonna win in a bet the person whose reasoning to be right versus the person whose reasoning to be accurate that's why when I say to you do want a bet it takes you out of the right mode right into the accurate mode because everybody understands like if I'm gonna bet on it I better be reasoning to be accurate so he told me you have to be reasoning to be accurate what you're doing right now is reasoning to be right you're sad you lost it's not your fault blood will blood that doesn't that doesn't help anybody but if you have a question if you want to know whether you accurately play the hand let's talk about it so he told me that that I have to think about accuracy if I'm talking to him he told me who's gonna be hold me accountable to that because he did in that moment he was holding me accountable to the way I was thinking and he said you're gonna have to be open to maybe things aren't the way that you want them to be so I was gonna have to be open to diverse opinions and other people's perspectives so let's take those three things as a great group charter right you have to have a focus on accuracy a commitment to accuracy you have to hold each other accountable to these things and you have to be open to diverse viewpoints that seems like a pretty good charter and what's really wonderful about what Eric did for me in that moment we remain friends because I was thankful for this I was a little upset in the moment but I got over it I I saw I saw the value eventually but what he did for me was two things one is now I understood that when I went and talked to him he was gonna mentor me he was gonna teach me how to play so when I had hand questions I could actually deconstruct them with him and that was gonna be super helpful for me so that was a little bit of the help but the big help he gave me was when I went back and played on my own so when you're playing poker it's all on your own I I was processing what was happening at the table differently so I'm not processing oh I got unlucky oh I'm so smart oh I got unlucky oh I'm so smart because that's not good material for me to go back to Eric with right and I know later on he's gonna hold me accountable and if I want to have a really good discussion with him I have to come to him with good material so now it changes the way that I'm viewing what's happening at the table in the moment so that I'm looking for mistakes I'm looking for questions I'm looking for other people where other people might be out playing me where they might be winning because they're better than I am and I'm gonna go tell him and now this is what happens this is so great two great things happen from that thing number one I go back to him and I say you know what I think I really butchered a hand I think I really made a mistake and he gives me a pellet right so he wasn't giving me pellets for pressing the luck you know I just got unlucky lever right he's giving me pellets for that I think I made a mistake lever so that's really great so that now that becomes part of my self narrative right Eric Seidel wants to talk to me he thinks I'm smart right so I'm getting those I'm getting I would I'm not like I'm I practice mindfulness but I don't think I'm ever achieving like that you know on the mountain in Tibet not worried about what the world thinks of me I'm trying but I'm not gonna get there so Eric Seidel is giving me that right he's he's helping me to think better by giving me the right kind of pellet right that's number one and then number two again back to the I'm trying to be a monk on top of a mountain in Tibet I'm really trying hard for that mindfulness that that ultimate mindfulness practice but I'm kind of failing out and I'm just but I'm working on it I'm trying I'm practicing it but in the meantime if we go back to that tribalism and how does our tribe feel distinct from others think about what it did for me when I walked up to Erik and my group and I was saying I really think I made a mistake but when I'm walking down the hall of the poker room I'm hearing all these people go can you believe how unlucky I got yeah I got someone lucky - yeah oh that guy's an idiot did it it uh and that's what those tribes are how they're talking to each other so I got my distinctiveness in the mix - so it's like I got my cake and I got to eat it - which was kind of good and then in the meantime I'm doing a mindfulness practice and trying not to worry about any of that but until I can't you know until I get there I get to reinforce all of these good you know changes in cognitive style through what the group is giving me so I want to ask you one more question before I go the audience it seems like what you're you're talking about it's not just applicable to the poker world know it yeah I see the investment world I see family things like that where have you seen these skills kind of like spillover for you in your life oh my gosh everywhere so I want to say two things one is this these skills have definitely spilled over in all areas in my life and I'm terrible at them so the the thing is like these the this the way that we process information is so built-in that your goal is really to be a little bit better at it and if you're a little bit better at it it's gonna compound over time it's gonna really make a difference in your overall decision quality and the quality of the outcomes that you eventually have so I still mess up all the time trust me bad things have to me I go I can blue that oh so unlucky it's just that I catch myself hopefully more often and hopefully a little bit faster than I otherwise would but I know I'm still missing a lot of those I mean I'm still biased so that's number one but number two in terms of areas of my life like everywhere so like in the way that I'd parent my children in the way that I talked to my partner in the way that I the process of writing this book you know where you have to be like you're working with an editor and you have to not sort of like push off what they're saying because you feel so much ownership over the word words and you know better than they do it's like trying to have open mindedness and really use the editor in your agent and so and so forth is like that kind of truth seeking group that you're trying to come up with the best representation of your ideas and that's not necessarily all the ideas of how you think it should be presented that you have to be open minded to that you know just like I say like in terms of the way that I talk to my kids and I guide my kids like I you know every teenager that has I'm raised this comes back from so I can't believe I did so bad on the test that's because the teacher was mean and they gave me things that they didn't even teach in class and it was really hard and you can ask anybody right so like how are you talking to somebody about that and you know saying saying well with kids it's it's you don't want to be challenging them in the same way but you say okay that's I'm sorry about that so what are you gonna do about that like in the future how do you think you're gonna solve for that so really trying to get them to understand like what are the things that are under your control and what aren't so not in an Eric Seidel way but trying to say like okay how can you do better like how can you make it so that maybe you should go talk to the teacher for you know before the next test if you know they might put things on the test that they didn't teach so like that you know not so much disagreeing with them because I'm not you know they're not adults but trying to get them to for it's more on those things that are in their control but also just in terms of like when I was writing the book I was thinking like okay it's gonna be a certain amount of time that I'm gonna have to invest what you know am I gonna get what am I gonna get in return what does the future look like like I thought about the book could be a total flop I have to think about that in advance and is that going to feel okay for me is it gonna be worthwhile to have written the book even if nobody ever reads it so it's like really kind of everywhere and then you know really recognizing the other thing is I'm a big believer in Ulysses contracts because I really understand that as much as you sorry you're right I'm assuming so Ulysses contract is thinking in advance about places where you know you're gonna be really biased and it's gonna be hard to overcome this in order to kind of prevent yourself from acting irrationally so we don't always have a group of other people around us to help us but you can actually form a different kind of group which is a group of the past versions of you and the future versions of you to all kind of come to help gather and help the present version of you so the classic example is from Greek literature Odysseus was you know sailing along and he knew he was going to come to the island of the sirens and the sirens were these beautiful limps that sang on the shore and the song was so irresistible to the ears of any man that they would steer their ship toward the shore the shore was a rocky shoal that would break the ship apart sudden death so Ulysses which is the Roman name for Odysseus knew this and asked his crew to tie his hands to the mast of the ship so this is a really good example of forming a group that's different versions of you so Odysseus said future Odysseus is really gonna have a problem because future Odysseus is going to stare toward the shore so I'm going to tie future Odysseus's hands to stop him from doing that so I think there's all sorts of places where you can recognize that you might need help in the future like a simple example would be take a ride-sharing service to a bar that's a way to tie your hands you know for me with exercising I sign up for classes in advance so that I've got my schedule worked out and I feel that something you know that I have a commitment to that I only have certain kinds of food in the house you know these are all examples of Ulysses contracts and I'm a huge fan of those and I apply those across my whole life including actually in writing the book which was instead of instead of ever letting my editor set the deadline for me because that is more like someone telling me what to do which doesn't feel this as like a contract I said all my own deadlines so I I would tell her what day I actually would usually give her three days and a confidence interval around each one because you know well obviously because I wrote this book so so it would be with it but there would be a week that I would know that I had to turn it in and I would say you know Monday it's 60% on Monday and 80% on Tuesday and I'm 98% on Friday so so by setting that myself I was creating a contract and then you know I held to the deadline and actually got it the book in early because I was because I was applying Ulysses contract so I really do practice what I preach in the book I think it's an excellent concept and I'm just realizing oh my god I didn't see that because I was writing a letter or some way to house making a reference to the sirens in their song so that's excellent do we have any audience questions let's start here because I you were my poker opponent and then I'll move across the life yeah it's great my wife has a big family I come from China just not his you Korea party is very kind of anxious team from academia then they are now you change other causes so what kind of uh you talk about this isn't making right just for your life based on your life experience what kind of bad or big those kind of a when you make those theses and you know what you did you know it's naturally come to that they see there it's a sometime you really think about it your rents you know block some time and oh okay I need to change yeah that's basically hi so that's an interesting question so yeah I mean I have taken a lot of turns and I actually think that my life is a good example of really understanding kind of you know this luck versus decision quality so I would say that two big career careers that I've had have come out of some something happens that's either sort of on the face of it looks lucky or unlucky but it's definitely an intervention of luck and then I just say okay Here I am now let me think about what where I'm gonna go from here so it's not sort of sitting and wallowing and in the in the sort of luck element of it but just thinking okay I'm here now so now I all right that's done I've got to sort of figure out where to go from here so the first one would be this illness that landed me in the hospital for two weeks I still sometimes struggle with it but I you know I've got it's an it was very out of control at that time and now I've got it under control so it doesn't intervene in my life in the same way anymore but this was when I was really struggling with the stomach issue I land in the hospital for two weeks this seems like an incredibly unlucky thing that's happened to me and I say okay I need money what am I gonna do I don't have my fellowship anymore it turns out that when you leave graduate school and take time off that and the National Science Foundation doesn't still give you money so all right I got to figure out what I'm gonna do and that's when I started playing poker and then when I was playing poker I discovered okay like I'm pretty good at it and I really enjoy it and it seemed like this really cool like collision of what I was studying in academics and sort of real life right and so and and at that point I said okay I'm just gonna delay graduate school and this is what I did once I started playing poker I didn't necessarily think there was something else I was gonna do then I got asked to give this talk in 2002 it was great and I still didn't think and that was a matter of luck because Eric Seidel actually asked God ask to get that give the talk not me and he hates public speaking so he referred me this had nothing to do with me I wasn't seeking it out right this was not a decision I made I get offered the talk I say yes I'd give it and I'm still thinking oh that was fun except that the person the people that I gave the talk to started referring me out at which point I said okay now I've got it like I've got a choice I could maybe I could do this as well which I did so and there's been all sorts of there's other points in my life I almost became an options trader a little while after that but then poker got on TV and I thought that's kind of interesting maybe I'll try that and I think that because I think probabilistically I end up I'm just less likely to stick with the status quo I'm more likely to say okay it's a new decision do I want to switch grocery lines and so I think that's sort of how I end up looking like I'm kind of here or here or here it's just ok luck brings you to certain places and I'm perfectly Oh fine fine saying here's the status quo and here is this new thing that I can do what can I do it maybe I can do them both at the same time I think I tend to be more open-minded maybe to that because that's the thing that poker really teaches you it's like the best laid plans right you still lose the hand and so it forces you to think probabilistically you have to be thinking this way or you will not succeed and I think naturally it spreads out here here's the actual funniest example that I can give you of how poker players just really think this way so I when I went to go on my first date with the absolute love of my life my brother and brother-in-law who both knew the person I was going on the first date with immediately set a market so so they started bidding we'd never been on a date we had been friends for a long time but we've never been on a date so they they start bidding and my brother-in-law ended up buying 23 so somebody said to me aren't you offended that they started making a market and I was like no I just I just can't believe my brother-in-law bought 23 like we'd never been on a date so I think that that shows you it's like just this way of thinking about the world just sort of a you know attaching numbers to it and and that's why I think that I sort of you know I hope that answers your question so our next question is online okay you've talked about the feeling like an imposter or outsider some poker tables what are some of the ways the ideas in your book can help address the imposter syndrome sensation or counteract overconfidence yeah that's that's actually really good so I think the overconfidence piece is a little bit easier to to deal with because I think that if you have a really good decision group they're gonna check your overconfidence so that's gonna be part of the conversation all the time is is are you feeling overconfident impostor syndrome you know it's it's a hard one I mean I've feel it all the time I feel it about this book I feel it when I get up and give talks it's like you know why am I even opening my mouth and I I think that again it's like get other people to talk to you you know because what they can do is they can often see things more clearly so I've actually had conversations where I said you know oh I just feel like this I can't believe it I'm such an imposter I don't know what I'm talking about it and people will say like people keep hiring you for speeches they'll actually go through it like you know they refer you out people are buying your book they're obviously you know and having other people to kind of look at you in a more objective way to get you out of your own head I think I really feel like that's the way to do it and then what happens is it's like I highly recommend everybody read the power of habit because it really shows you how to change your habits but not just not just the habits of like going to the gym like your habits of mind and the way you think and what ends up is you know remember I said when I had that conversation with Eric Seidel now when I went to the table I was thinking about what Eric Seidel might say later and so when you have a really good decision group that's checking your bias whether it's over confidence or under confidence or impostor syndrome the group gets in your head in a really good way so that then when I'm sort of on my own and I'm like trust me when I leave this room and I'm in the car I'm gonna think about what a bad talk I gave so maybe I did I don't know I just so now I'm in my head okay so anyway I when I do that hopefully I have a group of people that I can sort of offload that to and they can offload things like that to me and hopefully they're talking to me I mean not physically but that I'm sort of running that conversation you know because that's really what happened at the poker table was if I if I was processing the world in like a way that was irrational I would run the conversation with Eric or my brother or John Hennigan or whoever might be later where they were telling me like oh come on and that would help me when I was kind of on my own so I think it's about how are you how what are the rewards you're getting and how are people around you helping to sort of shape you know your those those habits of mind and I think that's really the best way because I think it's just the thing is I think it's so hard to do on your own I think we all really need help with this you know and I think another another mistake we make is we think well if I know about it like if I know about impostor syndrome or I know about overconfidence and I'm really smart problem solved in the book didn't you mention that the people who have the highest IQ scores are the ones you're most susceptible to yeah so being actually being high IQ correlates positively with motivated reasoning so it right and you can think why that is right it's like how do you slice and dice the info like if you're cognitively agile you're gonna be able to slice and dice a story that sounds a lot more convincing to yourself and to others about why everything you believe is true and why every prediction you ever have is a hundred percent so yeah so I'm smart and I know about this now is not necessarily a solution and like I say like I I'm telling you I'm gonna walk away from this I'm gonna think about what a bad job I did I'm not I don't want you guys to think that like you you you practice these things and somehow it's ever solved it's not it's just do I catch myself doing it more often do I do it less and those just doing it less and catching it more often it's huge so instead of like berating yourself for all the times that you screw up it's like you get to celebrate that I did it one fewer times you know like it's it's such a more self compassionate way to live like and I'm really like I'm trying to get there but you know yeah so I think you had a question yeah oh sorry we're we supposed to end it with no we're good keep going okay short question what's your strategy for rock-paper-scissors Oh short answer is throw scissors so just kind of understand so most people threw a rock first and then most people kind of know you throw a rock so so what happens is well depends if you're against somebody who is a total newbie so you know they're gonna throw a rock then you should you know just throw paper but but if you're against somebody who you think is a little bit smarter than that they're going to throw a paper on you so you do a pretty good job throwing scissors thank you and if they're really good they throw scissors and you tie so it's not a big deal one of the things that you said at the beginning of the talk like really stuck with me about like talking about your opinions and for in terms of bets and challenging that how do you do that when something isn't quite as quantifiable as like the number of representatives in the House in November sure so everything can be quantified the chances that you because every everything is probabilistic like the chances that you meet the love of your life tomorrow or that the person that you stay with you'll still be within 10 years or whether you're gonna like the dish that you order at a restaurant or literally that you know at literally everything is quantifiable now do we have enough data to get to flipping heads or tails where we know we know 50 percent of the time that you're gonna flip heads no I is there some objective probability for anything short do we have access to it absolutely not but we can sort of come up with a range now so what people will say to me is often well it's not really quantifiable well of course it is it's just that it's harder it's there's more unknown information it's harder to get to we don't have as much data so what I tell them is look let's say that I were to ask you some question about well what do you think the chances are that this belief that you have is true and the best you could do is somewhere between 20 and 80 percent right that's a really wide market but you know what it's not as wide as 0 to 100 like at least you're trying that's the thing so even if you say I don't know I can narrow it down to between 20 and 80 percent that's not a failure that's a success because you're trying and then what that does is because you've you you say well okay it's 20 to 80% man I'd really like to try to refine that you'll now start to go ask people like you're gonna go start asking for information you're whereas if you just sort of leave it as like whatever you don't actually ever try to get better at it so now maybe you go and you start researching and you you know what I make a joke and literally every single talk that I give I say what this does for you is it makes you take out and look at Google you guys will be very having it but literally it's a joke in every time in every talk that I give it makes you want to Google things so but it makes you want to Google things so so now you start googling and maybe you you okay now I'm narrowed down its between 72% and right so like that's what happens you know now it's 72 28 so and now you're a little bit narrower than than 8020 and that's all we're trying to do but if we don't think about things probabilistically if we don't recognize that what's gonna happen in the future is probabilistic that we can't guarantee it we are not playing chess if I play better than you it does not mean I win not in life only in chess right that if we don't do that then we're gonna be really surprised when a bad outcome happens we're gonna over react to it we're gonna start changing strategy when we shouldn't we're gonna reinforce strategies that we shouldn't we're gonna always be reacting instead of planning ahead instead of being proactive because we're not understanding that well other things would happen then the thing that we think you know so I think just just try like cuz ever literally everything's quantifiable where were you at on Black Friday in 2011 and what are your thoughts on regulating online poker oh okay that's an interesting question so where I was at actually I remember this I was in my bedroom and I got a call you're you should open up the internet and you should look so Black Friday is the DOJ came and down and shut down all of online poker it was a very bad day for poker everybody was really mad it was it was it's like it's really hard like in a lot of ways the poker community hasn't recovered from it so it was it was kind of interesting because I played in the 90s when in poker money was not raining down from the sky you know there was a very small group of people who were playing and then he'd sometimes have you know someone who came in and would play and you know you were it was really like you were it was called grinding and then online poker happened with the closing of poker on TV and the poker economy just boomed I mean I mean boomed it was really like just money raining down from the sky and then the DOJ on Black Friday now it all contracts and that was really hard I think it was less hard for the people who had been playing in the 90s because I think we sort of saw the luck in what it happened during the 2000s a little bit more but it's like whenever anything goes through like a big recession and I would call it a depression it's painful right like it's super super super super painful you know and in terms of whether there should be regulation I'm a you know I'm somewhat small government but I'm also you know I like people to have protections so I did testify in front of Congress twice about regulation and I tend to be more on the sort of free for your side of that discussion you know that you should let adults choose what they want to do and then obviously make sure that there isn't fraud occurring right so you should always have protections against fraud excellent what resonated with me you questioned the status quo yeah I think it's a it's great because it gives you a lot of opportunity to see your options in life all the time itself just going what you have but also I have the feeling it's a it's a risky business you know you can like invest a lot of time and always questioning your status quo and instead of focusing on what you're doing and like trusting your decisions you made in the past and like how do you find the balance you know and you seem to have been going good place of it so do you set yourself you know mine stores oh I'm questioning where I'm at in life next month from now or do you like two minutes later when you're on the wrong line I mean you know how much time do you invest in them yes that's about two I think there's a couple I think certainly from a business standpoint we always go to you know there's always strategic planning sessions and I think that when we do that it's always around like what's the new decision and what doesn't get included in there is well what about sticking with what we're doing so I think that we tend to get involved in strategic planning decisions generally when there's something wrong and you should actually set a calendar for all right let's let's start thinking about what are our goals regardless of whether we think something is wrong because maybe there's a better way to do it and and actually working backwards is but so something called back casting and pre-mortem x' is the best way to do that so imagine what your goal is think about that you've actually reached it this is a back cast and then think about how you got to the how you actually did it so it's three years from now and you know we accomplished x we do that but then also a pre-mortem is its rears from now and we failed you know how did that happen and I think that allows you to see the landscape a little bit better what I would say in terms of you know are you doing it too much or are you doing it too little there there's two things that I want to say about that one is that I certainly don't think that every decision that you're ever supposed to be made is suppose it's supposed to be done as a deliberative process if you did that I wouldn't be able to get from here to there right so a lot of stuff that's happening is certainly more gut and it's more in the moment and I think that that's totally fine as long as you hold your intuition accountable to a deliberative process at some point so when somebody says to you well why did you do what you did your answer should never be because my gut told me to and then when they say okay that's great but why it shouldn't be because I told you because my gut told me to if you can't teach it to another person there's a problem so that being said I think intuition can be very good as long as you're checking it occasionally but in terms of whether you should have a schedule I think once you start thinking about the world more probabilistically you're naturally examining those things and it's happening relatively automatically and you do end up sticking with the status quo a lot it's just that you're sort of seeing these opportunities over on the side more now what's interesting is that when you do that you don't just overreact and like just because things aren't going well right now you you're actually less likely I think to change course because I think that a lot of our changing course is a reaction to the way that the status quo is is now it's a reaction to having a bad outcome and a lot of not changing courses that we just happen to have good outcomes so I think that what happens is that when you start thinking in this way more probabilistically even if you have good outcomes you are you're always sort of working with your decision group of like well is there a better way that doesn't necessarily mean you do it it just means it's it's in your head so that you're you're more likely to sort of see the opportunity when it comes your way but what's more important is you're less likely to overreact to a bad outcome happening where you actually change course too much and I would actually say that interestingly enough that that second problem is what the big problem the bigger problem is it's not so much that if you think this way you're gonna be going left right left right left right it's that you're not gonna be just going left because bad things are happening all the time you're gonna be setting back into a more deliberative place on those things that's actually gonna either cause you to make sort of incremental changes that are gonna be really good it's gonna cause you to be able to anticipate or think about paradigm shifts for yourself a little bit better and I think it's gonna actually give you a smoother ride much less much more so than a less smooth ride I don't know let's see now now you might go and say no she should actually get into that limo and I do that does that answer your question yeah I know I'm four I'm all for status quo yeah well yeah so I think it's like again it has to do right it has to do with separating those those that don't pat yourself I sort of say don't get so down on yourself for your bad outcomes and don't pat yourself on the back so hard for your good ones so like here's a really great way actually that implement within your own life it's like once a week have a tier team get together think of each person presents like the best success that they had that week and everybody talks about like they have to offer up the five things that they did wrong along the way to that success so it's not about changing course right it's about it's more about refining about always trying to do a little bit better and I think that when you have that goal in mind I think you do a lot better because otherwise it's what makes us make these big left turns I mean I mean I've got a couple in my life but you know what makes us make these big left turns I think is when we're reacting to the way that things are we're just reacting so when you're trying to separate those two things out and always be trying to focus on process now what happens is you're less likely to be making these like 90-degree changes I think I'll have an online question what is your approach to the mental game of poker when running bad when our emotions can potentially influence our thought process can you compare that to the advice offered by Jared tender and his popular mental games mental game of poker bucks okay so I haven't read those books so I can't make the comparison but I can answer the first part of the question okay so there's something in poker called tilt tilt is when our emotions get lit up that shuts our frontal cortex your frontal cortex down and we start making really bad decisions so I I say about poker has this really interesting quality of your exchanging chips you know every single hand and so it's letting you see that there's risk in every single decision so that's the good thing the bad thing is it's allowing you to get caught in the path right so if you think about it I mean obviously what we'd really like is to think of ourselves it's like the Berkshire Hathaway stock right we want we want a general upward trend to our happiness stock over our lifetime and if you look at birth your Mercer Hathaway from the moment of its inception it has that general upward when and it looks like a pretty nice little ride but if you look at it on a given day it might have a sharp downturn and in poker because you've got this exchange happening all the time you end up doing what's called ticker watching and you can really see those down swings and you can really feel those up swings and so you start getting yanked around in this kind of like path-dependent you know your very path dependent in terms of where your emotions are so and this is generally a problem in life right so as an example like let's say that you're on the side of the road and you have a flat tire and it's raining and it's really cold and you don't have a jack and you call triple-a and you know they take forever and so you're just cursing your life right like you're just thinking like I have the unhappiest life I can't even imagine it why do these things always happen to me oh do you think it matters if you got a promotion three days before are you like now hold on a second let me actually take the average of my happiness over the last three days no yeah it's like literally what happened three days ago doesn't matter you're only you're really deep down in that moment so this actually gives us a clue to how we deal with this and it's part of how poker players deal with it so if I were to say to you okay let's say that you had that flat tire a year ago do you think that that would have affected your happiness one bit over the course of the last year that one incident I actually to tell you the truth I actually think it would I think it would uptick I think it would cause an uptick in your happiness because it's a really good cocktail party story interesting articles that I've seen a lot of kind of current events and thinking about it in terms of resulting or hindsight bias or self-serving bias or motivated reasoning or whatever it might be to really kind of look at how what are things that are happening out in the world that can help us to try to figure out how to be better decision-makers so like this week for example I talked a little bit about the NFL Draft and decision making around the NFL Draft I also talked about a really interesting study that was done on inducing intellectual humility in high school students and how that actually increases their ability to learn to really get that into that sort of mode of I'm not sure so that was in there I talked about something from near I all on the peak end role and how you can hack that in order to make your memories better and your experiences better at least your memory of your experiences better so that you know and then I'll also talk about things and that you know I not in this particular one but like last week I think I talked about current events things that are happening on politics and that kind of thing so hopefully people will subscribe to the newsletter you can actually find archives of the newsletter on Annie dookhan so you don't have to buy it until you actually test the product so you can you can go see the mornings during your podcast I think you do a great job doing that well I appreciate that so I'm not so I've had that suggestion I'm thinking about it and right now because of the launch of the book I'm doing a lot of podcasts yeah and so I'm kind of enjoying that at the moment I've done some really really fun ones and one and actually it's exposed me to a bunch of podcasts that I didn't know about so that's been actually super fun is that I'm getting to find out about some podcasts that I wasn't personally aware of I love podcasts because I think I mean some of them are obviously like you know your 15 minute deals but but the ones that are more like this where you get to go you get to do a deep dive and then what I fine that's so amazing about them is that I rarely come from a away from a podcast where I haven't learned something I mean I'm always not just one thing either like so much where you get to sit and talk to somebody who's obviously really smart and really engaged and see what their perspective on your product is it's so cool and it's just like such gratitude for how it changed like you you come in and you're like well this is my this is my work product and then you come away going oh my gosh what a like I didn't think of that and I was so challenged and that makes me think about it in a different way and oh this relates to this in a way that I hadn't thought before so I'm just enjoying that process right now and then we'll see I don't know we'll see what the future holds in terms of anything else excellent well if you haven't already grabbed her book it's called thinking in bets making smarter decisions when you don't have all the facts if we can give her it please a huge round of applause [Applause] you