The Simple Tools for Making Better Choices with Annie Duke
Annie Duke is an author, corporate speaker, and consultant in the decision-making space. Annie’s latest book, How to Decide: Simple Tools for Making Better Choices, is available now. Her previous book, Thinking in Bets, is a national bestseller. As a former professional poker player, Annie won more than $4 million in tournament poker before retiring from the game in 2012. Before becoming a professional player, Annie was awarded a National Science Foundation Fellowship to study Cognitive Psychology at the University of Pennsylvania.
In this episode, we discuss how to think about luck, the hidden information you can find in any situation, and why uncertainty is key to making good decisions!
The intersection between poker and cognitive science
Poker is a high-stakes decision-making framework…
Thinking about luck, hidden information, and uncertainty is the KEY to making good decisions
Outcome quality and decision quality are not correlated at 1.0 - they are related, but they are not directly correlated.
In life, even the simplest decisions are fraught with lots of hidden information and lots of luck.
Lots of GOOD outcomes are a result of BAD decisions.
Pay a lot of attention to good outcomes, and figure out what was luck and what was skill.
How do we evaluate luck, skill, thinking, and decision making in an uncertain environment like coronavirus?
When there is a lot of uncertainty.. focus on making “reversible” decisions first.
Second.. make decisions that have the lowest level of consequences first, to see if you can get more information to make a bigger decision
Ask yourself: What are the signals that would cause me to change my model or change my opinion about this?
How do you deal with politicized information?
Making and identifying a “Free roll” decision is one of the easiest and best things you should do.
Identity becomes a HUGE problem in our ability to process information in our decision making.
How to identify decision making free rolls - “if the worst case happens, am I worse off"?
How do you create a shared framework of decision making and making better choices?
The only way to update your beliefs is to go out and put them to the test and to collide with things you disagree with.
Homework: Figure out how to elicit other people’s true opinions, don’t poison them with your opinions first.
Thank you so much for listening!
Please SUBSCRIBE and LEAVE US A REVIEW on iTunes! (Click here for instructions on how to do that).
Wish you were in early on some of the best performing IPOs of 2019 and 2020?
OurCrowd investors have benefited from OurCrowd companies IPOing — like Beyond Meat — or being bought by companies like Intel, Nike, Microsoft, and Oracle.
Today, you can join OurCrowd’s Investment in Memic — a much-needed innovation in the rapidly-growing, multi-billion-dollar, robotic surgery market.
Memic reports that with their robotics surgeons are performing safer surgeries and patients are healing faster and have less scarring.
You can get in early on Memic and other unique opportunities at www.ourcrowd.com/science. If you’re interested in investing, you need to join OurCrowd.
The OurCrowd account is FREE, just go to www.ourcrowd.com/science
What interferes with your happiness?
I know better than most that your thoughts and feelings aren't always true and can hold you bak from achieving your goals.
That's why I'm excited to share with you our sponsor, BetterHelp. BetterHelp is offering 10% your first month when you go to www.betterhelp.com/sos and sign up.
BetterHelp isn't self-help, it's professional counseling. You'll get timeline and thoughtful responses from counselors and can schedule weekly video or phone sessions. All without ever having to enter a waiting room.
BetterHelp can provide you with professional counselors who are specialized in depression, anxiety, stress, relationships, grief, and much more.
Try them today and get 10% off your first month by going to www.betterhelp.com/sos!
You've got to check out this incredible online community Matt and I have become a part of. It's the ultimate knowledge hub from The Hustle, called Trends.
Trends allows you to interact with and learn from a community of industry leaders to be the first to know what emerging trends are coming next.
No matter what you do, this information is incredibly valuable! They also host weekly live lectures on tons of topics from growth strategies, SEO, and more!
Right now you can get your first two weeks for just one dollar at www.trends.co/success!
Check it out and join Matt and I today!!
Want To Dig In More?! - Here’s The Show Notes, Links, & Research
General
Media
Constant Renewal - “Annie Duke: How To Make Decisions Like A Poker Player”
Texas CEO Magazine - “How to Hedge Against an Unseen Enemy: A Conversation with Annie Duke”
Advisor Perspectives - “A Champion Poker Player on How to Make Better Decisions” by Dorothy Hinchcliff
Berkeley Haas - “New Thinking in a Pandemic: Don Moore & Annie Duke” By Laura Counts
Marginal Revolution - “My Conversation with Annie Duke” by Tyler Cowen
[Book Review] Grounded Curiosity - Book Review – Thinking in Bets, by Annie Duke
Barron’s - “This Famous Poker Player Can Help Investors Play a Tough Hand” By Leslie P. Norton and Bonnie Bennett Slater
The Moth - Annie Duke Storytelling
Behavioral Scientist - “Redefining Wrong in Poker, Politics, and Beyond” By Annie Duke (2018)
[Podcast] CAPITAL ALLOCATORS - Annie Duke – Decision Making in a Crisis (Capital Allocators, EP.132)
[Podcast] Unmistakable Creative - HOW TO MAKE BETTER DECISIONS BY THINKING IN BETS WITH ANNIE DUKE
[Podcast] Medium - Conversations with Tyler - Annie Duke on Poker, Probabilities, and How We Make Decisions (Ep. 99)
[Podcast] Farnum Street - Getting Better by Being Wrong with Annie Duke [The Knowledge Project Ep. #37]
[Podcast] The Jordan Harbinger Show - 40: Annie Duke | How to Make Decisions Like a Poker Champ
Videos
Annie’s YouTube Channel
Productivity Game - THINKING IN BETS by Annie Duke | Core Message
Talks at Google - Thinking in Bets | Annie Duke | Talks at Google
Big Think - 'Resulting': Don’t mistake a bad outcome for a bad decision | Annie Duke | Big Think
TEDxTalks - Risk Schmisk | Annie Duke | TEDxGeorgetown
MoneyNeverSleeps - Annie Duke - Making Better Decisions, Part 1
Books
Amazon Author Page
How to Decide: Simple Tools for Making Better Choices Sep 15, 2020 by Annie Duke
Thinking in Bets: Making Smarter Decisions When You Don't Have All the Facts by Annie Duke
Heads Up Tournament Poker by Vanessa Rousso, Annie Duke, and Josh Doody
The Middle Zone: Mastering the Most Difficult Hands in Hold'em Poker by John Vorhaus and Annie Duke
Decide to Play Great Poker: A Strategy Guide to No-Limit Texas Hold Em by John Vorhaus and Annie Duke
Annie Duke: How I Raised, Folded, Bluffed, Flirted, Cursed, and Won Millions at the World Series of Poker by Annie Duke and David Diamond
Misc
Episode Transcript
[00:00:04.4] ANNOUNCER: Welcome to the Science of Success, the number one evidence-based growth podcast on the internet, bringing the world's top experts right to you. Introducing your hosts, Matt Bodnar and Austin Fabel.
[00:00:19] AF: Welcome to another episode of the Science of Success, the number one evidence-based growth podcast on the Internet with over 5 million downloads and listeners, just like you, in over 100 countries. I’m your co-host, Austin Fable. Today, we've got an incredible guest on the show who's been with us in the past, Annie Duke.
We dig into a ton of great information, including how to think about luck, the hidden information in any decision you may make and how uncertainty is actually the key to making good decisions. Before we dig in, are you enjoying the show and the content we put out each week for you? If so, there are two incredibly easy, yet tremendously impactful things you can do for Matt and I. First, leave us a quick five-star review on your podcast listening platform of choice. It helps other listeners like you find the show, so we can reach more people with this great knowledge.
Next, go to our homepage at www.successpodcast.com and sign up for our e-mail list today. Our subscribers are the first to know about all the comings and goings of the show, but also, you have access to exclusive content you won't find anywhere else. Specifically, when you sign up, you're going to get our free course that we spent a ton of time on called fittingly, How to Make Time for What Matters Most in Your Life.
Are you on the go? Maybe you're working out, driving in the car, well good for you. Sign up for our e-mail list, but just by texting the word ‘smarter’, that's S-M-A-R-T-E-R to the number 44-222 and you'll be signed up today. If you are driving, do not text while driving. Pull over and send the text, just to be clear.
If you haven't already checked out last week's episode with David Kidder, we dig into what moments of disruption can mean for anyone and how to be sure when making any business decision. On this episode, we interview an incredible guest, Annie Duke. We talk about decision-making on many, many levels.
A little bit about Annie. Annie Duke is an author, corporate speaker and consultant in the decision-making space. Annie’s latest book, How to Decide: Simple Tools for Making Better Choices, is available now wherever books are sold. Her previous book, Thinking In Bets, is a national best-seller.
As a former professional poker player, Annie won more than 4 million dollars in tournament poker before retiring from the game in 2012. Prior to becoming a professional player though, Annie was awarded a National Science Foundation Fellowship to study cognitive psychology at the University of Pennsylvania.
We had a great conversation. Annie as I mentioned, is a repeat guest. She's always great to have on the show. Her insights are incredible. Without further ado, here's our interview with Annie Duke.
[00:03:04] MB: Annie, welcome back to the Science of Success.
[00:03:06] AD: Thanks for having me.
[00:03:08] MB: Well, we're really excited to have you back on the show. Austin, I’m sure you're excited to have Annie back on as well.
[00:03:14] AF: Yeah, I’m super excited to have you on, Annie. I wasn't on the mic when you came on for the first run, so super excited to dig in with you, especially in light of everything going on in the world and the new book, of course.
[00:03:26] AD: Well, I’m happy that you've been allowed to have a voice, to be able to chat. I’m excited that this is going to be a real conversation. This is good.
[00:03:36] AF: I love it. I love it.
[00:03:38] MB: Yeah, we're super pumped. I mean, to jump in and start out, I mean, I think I said this in our last conversation too, but I mean, for me, Thinking In Bets was probably one of my favorite books I’ve read in the last three or four years. I mean, I’m an amateur poker player. Not very good, but I like to play and I’ve learned a tremendous amount about life and decision-making from poker.
I’ve recommended that book to a half dozen people or more, just about this is a great framework for thinking through decision-making. How did you go from that to your new project and really focusing even more on how we can make better decisions?
[00:04:17] AD: The natural evolution of the way that I think about decision-making, I think you can see this in the relationship between the two books. I started off my life strictly as an academic, doing my PhD work at UPenn. Then, I transitioned into poker and was really deep and focused into poker. Then around 2002, I started thinking this very explicit way about the way that poker and cognitive science can have a really interesting conversation with each other, where there's all this amazing work in cognitive science about how do you learn, how do you make a good decision, cognitive bias, what are the errors where we go wrong.
Then poker is really this high-stakes, really fast decision-making environment in which all of the science is being brought to bear at that table, both sides of the issue, which is what are the ways in which that environment causes your decision-making to go wrong that you would also see in the science.
Then, also something that I felt was maybe a little bit lacking in the science, which was how would you actually solve for some of these things? Because at poker, if you don't solve for them, you go broke. There's an immediate consequence to it. You just lose your chips. I started thinking about how those speak to each other and then, that really is what resulted in Thinking In Bets, which was this book that was a little bit of how, but lots of why, about why is uncertainty so important, why is thinking about luck and hidden information so incredibly important to creating a good decision process. What does that mean about how do we actually learn from experience? How do we understand what the outcomes of our decisions actually mean when you could lose, because you played poorly, or you could lose because you played quite well?
If you got unlucky, or vice versa, you could win, because you played really well. Or maybe, you could win, because you played really poorly. I just realized it was very hard to work backwards and started thinking about the solutions that that poker really offers you to that a little bit. Lots of why about uncertainty. Let's really think about uncertainty, why you need to pay attention to it. Then a little bit of how would you actually solve for this and create a good decision process.
After I had written that book, I just felt I really wanted to ground these ideas much more and offer somebody a book that would actually walk them through how do you think about learning in a better way? How do you use the feedback that the world gives you in a way that can actually help you to learn better? What does a good decision process looks like? How do you figure out when you should take a lot of time with a decision, or when you can punt it and go fast?
The last piece of the book is really just how do you think about the conversations that you're having with people in order to really help advance your learning, as well as theirs? Because this was a really big issue in poker, was that a lot of the work that you were doing as a poker player was talking to other great poker players, and learning how to have those conversations, which is actually not natural, in a way that could actually help you to learn and overcome these cognitive biases and these interferences in learning is actually really important, so I actually devote a lot of time.
I think about Thinking In Bets is a little bit of a why in explaining this conversation between poker and cognitive science. Then this book is just how do you actually do it? Let's really get down to brass tacks and figure out how to ground it.
[00:08:13] MB: I love that. I think it's such a great framework. I definitely want to dig into this notion of how do we create almost a shared framework, or a shared vocabulary to explore these concepts amongst people who want to make better decisions, or people who want to improve their thinking. Before we even jump into that, you touched on something a minute ago that to me is such a critical insight, which long-time listeners have heard me say this, both in your previous interview and many other interviews is poker is such an incredible crucible for forcing good decision-making, because if you don't learn how to make better decisions and assess your own decision-making, you go broke. It's a very unforgiving game and that's why it's such a beautiful teacher.
To me, that that lesson that you can make a bad decision and have a good outcome, I see that pattern play out so much in real life, where people have an absolutely terrible decision-making process and they get rewarded for it and it really creates a cascade of really, behavior that is extremely either risky, or they just don't understand what they're actually doing. I mean, there's so many manifestations of that. You touched on that idea of how luck can really impact our decisions and we don't often realize it. To me, that's such an important concept.
[00:09:35] AD: I’m so happy you brought that up. I mean, obviously, what you're talking about is it expresses itself in survivorship bias. All the books are written by people who just happen to have succeeded, not clear that they had a good process. What you said I think is – you're really hitting the point home of why I circle back to that topic a little bit. What I found was that after Thinking In Bets, of course, in that book I’m talking about that outcome quality and decision quality are not correlated at one.
They're related to each other, but that relationship is really going to express over the long run, over 10,000 coin flips, rather than for any particular outcome that you might observe that that's not going to be particularly good signal for a decision quality. You have to be doing something that's skilled, but also where really, the judgments aren't subjective. Meaning, that you have really perfect information, lots of information with very little influence of luck. The difference between chess and poker, in terms of that relationship between outcomes and decisions. If I lose at chess, I know it's because I made bad decisions. If I win at chess, I know it's because I made good ones. Not so in poker and not so in life.
In life, even the simplest decisions like going to a traffic light are fraught with lots of hidden information and lots of luck. When I go through a green light, I don't know what cars are coming in the other direction. I don't know if those drivers are drunk. I don't know if their cars are in repair. What happens is that I can go through a green light and get in an accident, or I could go through a green light and get through fine, and then we could do the flip side of going through a red light.
Now, the reason why what you said is really insightful is that – so I talk about this relationship. I can't tell you how many people after reading Thinking In Bets came up to me and said, “Annie, I’m so happy that I read that book, because I’ve really been beating myself up for these bad outcomes that I thought were my fault.” What I realized from that book is that oh, a lot of bad outcomes are probably pretty good decisions. It's just bad luck.
I was a little horrified that I had not expressed myself clearly enough to let people know that yes, while that is sometimes true, that the opposing concept must be true as well, that a lot of times when you have really good outcomes, it's actually the result of quite a poor decision process.
I feel the one side of the equation got expressed, or at least that's what people took out of the book, which of course is completely on me, but the other side of the equation didn't get expressed. Essentially, what I felt like happened was people said, oh, there's this disconnection between outcome quality and decision quality, so this is going to get me a way out of my bad decisions, because I can see that that this could be due to luck.
One of the arguments I really push in the first part of this book is that you have to be thinking about all four quadrants when it comes to the relationship between decision quality and outcome quality. You can have what I call earned rewards, which is a good decision, good outcome. You can have bad luck, which would be good decision, bad outcome. You can have good luck, which would be bad decision, good outcome and then you can have what I consider just desserts, which is you made a bad decision, you got a bad outcome.
I think the problem is that we don't actually explore those four quadrants equally. What I try to push for in this book is pay a lot of attention to the good outcomes. Really spend your time in there, because you want to know when it wasn't the result of a good process and you must explore that. Even more importantly, even if you think it was the result of a good decision, it doesn't mean that there wasn't a better decision to be had.
We don't want to rest on our laurels, “I made a pretty good decision. I got a pretty good outcome. I’m just going to leave that and not examine it.” As I tried to think about, well, why is it that we're not doing that, I thought about it in this way and you can tell me whether this resonates with you. If you have a bad outcome, you're already in psychological pain, because you're sad, because things didn't go your way.
If you look at the bad outcomes, either you may discover it was due to a bad decision, at which point you're still sad, but you may discover that it was due to bad luck, at which point, you all of a sudden become happy. You can't really lose anything by looking at the bad decisions, because you either stay in the same state you were already, which is sad, or you can get happy. If we look at the other side of the equation and we're really examining in a real way are good outcomes, in some sense, you can only lose to it, at least in the short run, not in the long run, but in the short run, in the sense that you're feeling pretty good about yourself, stuff went your way.
You closed a sale. Your portfolio increased in value. You won the poker hand. We could go through any examples. I’m feeling pretty good, because I have a good outcome. Now what happens when I examine it? Well, I may find that I made a really great decision. Well, okay. I’m already still happy. That didn't really change my state very much. If I find out it was actually quite a bad process that produced that good outcome, now all of a sudden, I feel bad, so I’m losing to that exploration in the short run.
Now obviously in the long run, it's good, because that's how you get better at things and then you don't repeat the same process over and over again, wondering why it isn't going right again. I think that that's why we're loathed to be actually looking at our wins and trying to figure out if we could have done better.
[00:15:29] AF: This episode of the Science of Success is brought to you by our partners at BetterHelp. That's BetterHelp. H-E-L-P. You'll get 10% off your first month by going to BetterHelp. Once again, that's betterhelp.com/ sos today for 10% off your first month.
Let me ask you, what interferes with your happiness? Is there something preventing you from achieving your goals? I know there have been plenty, plenty of times, getting too far into my own head has gotten in my way. Limiting beliefs, thoughts of doubt, it can all build up inside you and become paralyzing, or cause counterproductive behaviors to form. BetterHelp will assess your needs and match you with your own licensed professional therapist.
You'll get timely and thoughtful responses, plus you can schedule weekly video, or phone sessions all without dealing with one of my least favorite things ever, the waiting room. It's effective and more affordable than traditional online counseling and financial aid is available. Licensed professionals are on-staff, who specialize in things like depression, stress, anxiety, relationships, trauma and even more. Plus, everything you share is completely confidential.
In fact, so many people have been using BetterHelp, they are recruiting additional counselors in all 50 states right now. If you want to start living a happier life today, as a listener, you'll get 10% percent off your first month by visiting betterhelp.com/sos. Join over 1 million people taking charge of their mental health. Again, that's betterhelp.com/sos for 10% off your first month today.
[00:17:19] MB: Yeah, that's such a great point. It's funny, I wouldn't say it was necessarily the way that you frame things in Thinking In Bets that cause that confusion. I think it's really just the human brain and the fact that our biases always try to skew our perception of events towards whatever shines the most favorable light on us. Especially for somebody who's more of a newer initiate into the field of decision-making and hasn't really spent a lot of time thinking about and understanding how your brain can short-circuit and deceive you and trick you into thinking that you're smarter than you are and that you're right more often than you are, it's really easy to fall for that trap of finding an easy outlet that says, “Oh, I just failed because of bad luck, not because of bad decisions,” when as you said, really the vein that you can mine really richly is well, what about the good outcomes that happened as a result of poor process? How can you learn from those mistakes and improve your thinking and your decision-making?
[00:18:17] AD: Yeah. I mean, I generally agree, except the one thing that I would add is that I wrote in the book about how we tend to want to shine a favorable light on ourselves. I feel like, I more than most, had I really thought about it, I had a chance to recognize the way that that might be read.
Maybe I couldn't. Maybe I explained it pretty clearly and people are coming away with something just because of human nature that I wasn't saying. Maybe I could have said it better. At any rate, the good news is that I had an opportunity to say it better this time, I think. I’ve got specific exercises in this book that really have you focusing on good outcomes and trying to make some exploration about that.
One of the things just to add to what you said is it's not just that you may find out that you had bad process. You may find out that you had quite good process, but you would want to understand, process as a whole generally isn't good. There's some good parts of the process and some poor parts of the process.
First of all, you'd want to identify, separate that stuff out. The other thing is that as I – in poker, you can play a hand, have positive expected value. In other words, you're expected to win, given the way that you're playing, but that doesn't mean that you're maximizing your expected value. You could have a completely winning strategy, but there's another winning strategy that would win even more that's available to you. Because you're settling in and you don't want to feel like you made the second-best choice, which in the short run would feel bad, you don't actually discover that there's a better choice above that.
Because, it's basically almost never that you would be on the – what we would call the primary line of play. Meaning, the absolute best decision, if you had perfect knowledge and were the ideal decision-maker on earth, you're usually somewhere below that. We should always have that north star that we're striving for. That means that we have to be acknowledging that we may have a winning strategy, but that doesn't mean it's the best one.
[00:20:20] AF: Annie, I want to circle back real quick and talk about this idea of decision-making and how the right decision can possibly have a bad outcome and how a poor decision can have a good outcome, especially as it pertains to us in the world we find ourselves in today. As of recording this, we're in the middle of a pandemic. There's a lot of things going on where people are deciding whether or not they need to send kids back to school, how we do virtual learning.
In listening to the conversation thus far, it made me think, I’ve got several friends who are making the maybe, “bad decision.” They're going out to bars. The second restrictions get lifted, they're in rooms with hundreds of people, but they're all fine, healthy, thriving, no big deal. On the other hand, I’ve got friends of friends of mine who were super careful and had some pre-existing conditions, did not want to go out, wore masks everywhere and maybe in a moment, had to go to the store to get something, wore a mask, went and then they get a test and they're positive.
How do we go about assessing the risk in the world that we live in today and how we can make the best decisions when it comes to following something like, whether or not we're going to go out and be social, whether we follow certain regulations and things that pertain to the pandemic world we're in now?
[00:21:41] AD: Oh, that's such a great question, Austin. Let me just say that while the pandemic is certainly unusual, the uncertainty is actually relatively similar to the type of uncertainty that you might experience normally. Let me just explain what I mean by that. It's very clear when it comes to coronavirus that there's tons of luck involved. I mean, the right bat had to meet the right human for one thing.
[00:22:11] AF: Yeah, the beginning of it all.
[00:22:13] AD: Right. Then as you just pointed out about your friends, it's like, if you go to the grocery store, there's just a lot of luck involved, whether you walk through the aerosols of somebody who has it, or whether you don't. At that moment that you go to the store, if 3% of the people in your area are positive, how many people in that grocery store on that particular instance are going to be positive?
If 3% of people are positive, it doesn't mean 3% of people in the store are positive. Do you actually just run across somebody who is positive or not? On the flip side, if you go to a bar, do you end up sitting next to somebody who has it, or who doesn't? These are a lot matters of luck. Then there's also the problem of of course, the information landscape is shifting very quickly.
The things that we thought we knew about the virus in March are very different than the things we know today, just like the aerosols. In March, we thought it was spread through respiratory droplets, now we know that it can be aerosolized, so it can be sitting in the air. We know that now, that there's a big difference between you die, or you live and there's a whole world in between of being sick for a very long time, or longer-term damage to your organ systems, so we don't want to think about that as a dichotomy anymore.
This is all information that we're collecting as we go along. There's a lot of shifting right now about how contagious children are, for example, and that seems to change every single day. We can really feel the uncertainty so much right now. What I’ve heard people say is, “Well, I should hold off on making any decisions at all, until the environment goes back to being stable.”
The first thing that I want to say about that is what? What do you mean –
[00:23:49] AF: I was going to say. Yeah.
[00:23:50] AD: Back to being stable. I don't really know what that means. While this uncertainty is really heightened, it doesn't mean that there isn't tons of uncertainty at other times. What I try to say to people just to make the point is I just say, “Do you own stocks and bonds in your portfolio at the same time?” They'll always say, yes. I say, “Well then, you're admitting that the world is a little bit more coronavirus in its environment than you would to think.” That's a lot of the stuff that has to do with illusion of control, overconfidence, over-placement. Those kinds of things are all coming from this status quo bias, are coming from this illusion that there's less uncertainty than there actually is.
I just want to say, obviously, this is a complete tragedy that's unfolding right before us, but there's a lot of lessons to be learned from how to make good decisions when it comes to this environment that you can take out of it into your normal decision-making environment. I’ll just throw a couple out to you.
When there's a lot of uncertainty, you should always prioritize the more reversible decision. We can think about this from a logical standpoint. If we know the information is shifting all the time, so we're going to learn new things in the future. If we know that our decisions are only as good as the inputs, the information that goes into our decision, then what we can recognize is that while we might have to decide now, should we send our kids to school, should we not, for example. That sometime after we've made that decision, there's probably going to be new information that reveals itself to us and some of that information may make us wish that we had chosen something else to do.
When we're weighing two options or more, when we recognize a lot of uncertainty, we want to prioritize reversibility. If we think about the school decision, how could we prioritize reversibility? I’m not saying I have answers to this question. I’m just giving a framework for thinking about it. One is, is it easier to send people to school and then reverse and get them online, than it is to have kids online and then reverse and send them to school?
Whichever one of those is easier, you would probably want to prioritize that. We can also think about that in terms of reversibility of consequences. If kids are at home, for example, we know that they will have learning loss. We could think about how reversible is the learning loss, versus if kids are on in school, they may get their grandparents sick, who may die. They may get the teachers sick. There may be long-term damage from coronavirus. Those kinds of things that feel they would be less reversible than the learning loss.
Again, I’m not saying I have the answers to this, but that's how we want to be thinking about decisions, number one, is how can we be prioritizing reversibility? The second thing is how can we make the lowest impact decision possible? Meaning, something that's not going to impact us in the long run, in order specifically, to collect information.
Again, as a school, if we're thinking about the school problem, what I might say is there's a couple ways that I could think about how could I do some low-impact things first. One might be, it appears that the data looks better for under 10-year-olds than kids who are older than 10. I could send elementary school kids back, where I know that the impact of that is likely to be less, because it seems like, they don't spread it adults. Their parents are generally going to be younger, so that feels there's less risk there, so less of an impact. Even their grandparents are going to tend to be younger.
I could say, well, let let's send the little kids back and then wait and see on the rest, because then, we can collect information about what's happening at the elementary level. Another possible solution would be to say, well, there's a lot of school districts who are going to be going back. Why don't we wait a second, we'll start off online and then we'll use the fact that other school districts are going back in order to improve our information situation, because we're lacking information. That way, we can collect that information and see how it goes for those school districts prior to having to make the decision. That would be saying, we're going to start online and we're going to wait and see. That would be another way that you could do it.
A third thing that you can think about is exercising options in parallel. That would be an example of saying to parents, if you want to keep your kids at home, that's fine. We're going to have a really robust online learning system. Also, if you want to send your kids to school, that's fine. We'll actually try those two things in parallel. That would be another thing that you could do. There's all sorts of ways that you can implement this outside of coronavirus, in terms of thinking about that, but those would be three of the things that you should be really thinking about.
There's how quittable, or how reversible is the decision? What are the low-impact things that I can do that will give me the information I need before making the high-impact decision, or at least to improve the quality of that as the information is shifting? Can I do anything in parallel? That would be stocks and bonds is a way to exercise options in parallel. I’m not sure if things are going to go up or down. I could think about what the probability of those things are and that's going to determine the balance of my portfolio on stocks and bonds, ones which bearish on equities, one which is bullish on equities. There's lots of other different frameworks for thinking through this, but that would be one of the ways that you could think about those problems.
[00:29:27] MB: That's such a great insight and using the example, it's obviously very pertinent for everybody right now dealing with coronavirus, to think through and really apply some of these decision-making criterion, and to demonstrate that the world really is incredibly uncertain and there's lots of hidden information. It's really hard to make decisions. Austin, that was a great question. Thanks for throwing that one in there.
[00:29:51] AF: Annie, just for your sake and everything, I mean, we're not saying that – we're not trying to make a stance here. We're just looking at the current world and how decision-making can be done when we're faced with this uncertainty. I think it really dovetails too into talking about bias, because unfortunately, in the world we live in right now, it feels like, I mean, given the going back to school example, a lot of the people that are planning deep flags that actually have the power to impact lives, or making these decisions based on either political bias, or bias they may not be aware of, and it seems like a lot of these decisions are based on just bias in general.
I know one of the other things you discuss in how to decide is how do we identify these biases and I want to dig into that, but then also, how do we dismantle them, so that we know when we're making these high-impact decisions? We're not looking through the lens of bias, we're looking through the lens of fact and also, following the three steps that you laid out there previously.
[00:30:53] AD: Yeah, that's a really good question. I think I think that part of the problem and this is general, but it speaks to what you're talking about is that once we formed an opinion, once we've made a decision, we tend to interpret the world to fit with the decision, or the opinion that we have stated. In other words, we're trying to certify our beliefs all the time. When we make a decision, it becomes part of our identity, our belief system. If it turns out that information reveals itself later, that may suggest that the decision that we made isn't good.
I want to clarify what I mean by good, that you would have made a decision if you had the knowledge that you had now at the time. I want to just make that clear, because obviously, we're all making decisions where there's going to be stuff that reveals itself later that we wish we had known beforehand, but we don't know it beforehand. First of all, get comfortable with that and be okay with that.
Yet, we're not comfortable with that. What happens when the information appears in the world that would suggest that you might want to reverse course? This is part of the reason why I talk about really prioritizing reversibility, because it allows you, I think, to be thinking about the decision not as an identity choice, but as this is the best thing I can do at this moment, and I’m purposely trying to build in the ability to reverse course, because I recognize that I’m at an information deficit.
How can we think about how to improve the way that we think about that information as it comes in the future? Well, before you actually do the decision, really think about what are the implications? What are you thinking that the different outcomes are? What are the implications and explicitly state these, of what the states of the world might be in the future? What of those would cause you to stick with your decision, versus what of those might cause you to reverse course? Really do that in advance, so that you've got these signposts set up for yourself of if this were to occur in the world, I would actually change course.
The reason why you really need to do that is that once we've got a belief and part of that is we've made a decision, which is now a stated opinion and our opinions are part of our beliefs, that tends to dig a trench, so that when new information appears in the world, instead of looking at it objectively, we'll tend to pull it down into the trench with us. In other words, we'll interpret it to our favor, to support the decision that we already made.
By doing these things in advance and thinking about the decision in the sense of I want to figure out what could happen in the world that would make me change my mind and having that as a framework, that helps you to loosen up those, make the trench a little bit shallower. It makes it a little bit easier to climb out of it.
If we went back to the school decision, one of the things that we could think about with that would be we're going to open schools, but what are the number of infections that we would tolerate within the school before we would reverse course? Think about that well in advance and literally write that number down. That's something that we can do in general in our decision-making.
If you have for example, a particular model of the world in terms of your investment strategy, so we know that there are lots of different models, like there are people who are value investors, or trend followers, or they're growth, either investing in growth stocks, or whatever it might be, those are really just models of your decisions. When you decide on a model like that, it's really helpful to think, what would be the signals in the world which would cause me to change my mind, or cause me to alter my model? Do that really early in the process, so that you're not doing it as the information confronts you, because you've already thought about it in advance. I think that that's actually incredibly helpful for that piece.
In terms of what I would consider the politicization of information, which has become obviously, very problematic, where people aren't thinking about information on its own. They're essentially incorporating – They're thinking about what their political identity is. Then when someone who is a proxy for that political identity is speaking, they cease to evaluate the information and they take it as literally, just a signal of their identity.
I think that in June, there was a study that showed that Republicans were 12 times as likely to not wear masks as democrats. Now I want to be clear, that this is not a Republican-Democrat problem. This is an everybody problem, because Democrats also signal things all the time that cause Democrats to do things, where they haven't necessarily thought through the information either. This is an equal opportunity problem. I just want to make that clear. This example just has to do with the behavior of Republicans.
The interesting thing about masks, if we think, remember, I was talking about what's the impact of the decision. You have a decision between wearing a mask, or not wearing a mask. We can think this through. What's the impact of not wearing a mask? I think it's that you're uncomfortable. If you haven't brushed your teeth, you're smelling your bad breath. If it's hot out, you may be slightly uncomfortable. It may be a little bit hotter under that mask.
I’m trying to think of other really big downside impacts to wearing a mask, but I’m hard-pressed to find one. When we think about the upside of wearing a mask, well, we don't really know what it is. The experts say we don't know, but we know that it's good. In other words, we don't have certainty about what the positives are to that decision, but we know that it does help stem the spread. We know from other countries that have strong mask-wearing policies that it helps control the virus, even if we don't know by how much. It's just better.
This is a really good example of this framework that you can think through when you're uncertain. You can just look and say, outside of – let me just put the identity issue aside and who's saying it and let me actually examine the decision to think about what the difference is between the downside potential and the upside potential of the decision is. In this particular case, what we find out is there's basically no downside, but there's lots of potential upside, even if we're not certain about the upside.
That's actually what we call a free roll. A free roll is just when there's nothing to lose, but a lot to gain would be what a free roll is called. It comes from the 1950s and casinos, they would give people a free roll of nickels to go play the slot machines to get started. One of the arguments that I’ve heard about wearing masks is “Well, I’m not going to wear masks, because you can't tell me how effective they are.” What I’m trying to say is if you're thinking through this through a really clear decision framework, what you realize is it doesn't matter that you can't tell me how effective they are, because what I do have a lot of clarity about is what the downside is, which is almost nothing. I’m a little uncomfortable, because I have a mask on my face.
Therefore, that's something that we should just go ahead and do. There's all sorts of examples of free rolls in real life, applying to a college that's a reach would be a free roll. You don't really think you're going to get in, but nothing bad is really going to come from it. Asking someone out on a first date. They might say no. Who cares?
I think that this is generally a problem. You can take something like that and you can look at how much identity has gotten wrapped into it. What you can see is that identity becomes a really big problem in terms of our ability to actually process information in a rational way, in a logical way. Again, I just want to make it clear, this is a human problem. I’m not saying that Republicans are special in this way. Human beings are special in this way.
[00:38:44] AF: Sure. Free rolls are great, actually. It's funny, we look at that when we're reaching out to guests, I think even when we first reached out to you it's like, there's no downside. You're just going to either ignore us, or tell us no. Might as well ask everybody and their mom that we want to talk to.
[00:38:58] AD: Yeah. That's a really good example of a free roll as well. What's interesting is a lot of people won't reach out in that situation, because they're like, “Oh, I don't I think there's a really low chance they're going to say yes.” Okay, who cares?
One of the ways that you can identify free rolls and this is a really good framework for figuring out how to speed up, as they say, go fast and break things, is just ask yourself if the worst case happens, if the worst thing happens, am I worse off than I was before? In the case of reaching out to a long shot guest, if they don't reply, I don't think you're worse off than you were before. If they reply, you've got an awesome guest on your show.
[00:39:40] AF: Do you ever wish that you were in early on some of the best performing IPOs of 2019 and 2020? With our sponsor OurCrowd, accredited investors have access to invest directly, easily and most importantly, early. OurCrowd investors have benefited from OurCrowd companies IPO in just like Beyond Meat, or being bought by companies like Intel, Nike, Microsoft and Oracle.
If you're an accredited investor, you can join OurCrowd for free at O-U-R-C-R-O-W-D.com/science and review the current deals. No payments involved at all, until you decide to participate in a deal. As you refer deals, now here's one of my favorite parts, you have access to OurCrowd's investor relations team, who you can talk to directly on the phone about your personal investment goals.
Now the OurCrowd’s investor relations team have already done a ton of things to help you out. They've reviewed thousands of companies, invested hundreds of millions of dollars, closed investments on over 200 companies, chosen dozens of companies that have made exits and accredited investors can participate in a single company deal for as little as $10,000, or one of OurCrowd's funds for as little as $50,000.
Today, you can join Our Crowd's investment in Memic. Memic explains that their tiny robotics allow surgeons to be less invasive and safely perform gynecological surgeries, so women heal faster and have less scarring. Memic is a much needed innovation in the rapidly growing multi-billion dollar robotic surgery market. You can get in early on Memic and other unique opportunities at ourcrowd.com/science.
If you're interested in investing, you need to join OurCrowd. The OurCrowd account is free. Just go to OurCrowd. That's O-U-R-C-R-O-W-D.com/science to get started with your free account today.
[00:41:45] MB: Yeah, that's such a great framework. I mean, it's funny, because even you brought up a couple really good points talking earlier about just the decision-making process and the whole idea of putting identity aside and just looking at what's the upside of this behavior, what's the downside of this behavior. Regardless of what context you put it in, if a behavior has almost no downside and a potentially really high upside, it's a positive expected value decision, so you should make it. That's a great way to think about applying poker knowledge to any decision process.
You said something maybe a few minutes ago that to me is maybe the most – one of the most insightful things, I think you've said in this whole conversation, which has been filled with insights, is this notion of what signal, or what threshold, or what data point will cause you to change your model of reality, or change your opinion about a topic? If you're not willing to ask yourself that and commit to it on the front-end, then you're really falling into a major trap of all kinds of identity and bias and bad thinking really clouding your judgment and your decision-making.
To me, that's a really powerful question to ask yourself and to ask other people in any situation is what would cause you to change your opinion? What data? What information? Because if no data, or no information will cause you to change your opinion and I’m not talking about mask, I’m not talking about coronavirus, I’m not talking about poker hands, I’m not talking about anything. I’m talking about everything. Your opinion should always be willing to change if the data changes. I feel for many, many people I know, the answer is no amount of data would cause me to change my perspective.
[00:43:32] AD: I think that that's really true. I mean, I think even if it's a question of whether the earth is round, which is something that I believe quite deeply. I can think of exactly the data that would come to me that would make me change my mind. Now, I think that's very low probability to ever cross my path. I’m quite aware of what it might be. I think that this is something that really comes through in poker, because when I’m playing poker, obviously, I can't see your cards. What I’m really doing is building models of my opponent. That's what I’m doing all the time. That model of my opponent, I’m using for two purposes. One is I can only see your behavior in terms of trying to figure out what your hold cards are.
I have to build a pretty good model of you to figure out how your behavior might map onto a variety of different hands that you might be holding, which I can't see. I’m then looping that back and saying, given the hand that I think you might be holding, how do I expect that you would behave in the future? Meaning for Austin, I may believe that if he had a top pair and I raised him, that he was not the type of player who would fold in that situation, but maybe I think that Matt would fold in that situation.
I have to think about two people looking at the exact same hand of poker would react to those hands very differently. This is all this model building that I’m doing, that I’m then using to make forecasts about what I think you're going to do in the future. That's simply what you're doing at the poker table. Those are essentially, those forecasts that I’m creating, or those things that I’m thinking about that model, essentially, are those signposts that would cause me to change my mind.
What I’m thinking is in the negative space. I think that Matt has this particular type of hand. If I raise him, he should fold. If you don't fold, there are two things that I could be thinking about. One is that I’ve got your hand wrong and you actually have a stronger hand than I thought. The other is that you have the hand I thought, but you're perceiving that hand differently than I predicted. I have to now think about those hope. Maybe you show me your hand at the end and I can actually find out an answer, but I have to – now I immediately, because you did not act in the way that I expected, which was my sign post, I have to now start going back into my model and adjusting my model.
If you don't do that in poker, you will lose all your money, because you're never going to – you're going to think, “Oh, this player is this type of player.” Then you're not going to notice that that's not how they are. Because you're just going to be rejecting the evidence that's right before your eyes and ears and just say, “No, I know that I was right,” and continue along your merry way. A lot of poker players do that.
Particularly, I experienced that a lot as a female player, because people would have views of me as a woman and then they would stick to their view of me that I was a bad player, that I wasn't going to bluff a lot. For example, that I wasn't going to be particularly creative in my play. Despite the evidence that was coming their way that perhaps their model of me was poor, they would generally be pretty sticky in their model with me. That was something that I was on the good end of quite a bit. It's really important to bring that into your decision-making in general.
When you make a decision, really think about what are the things that could be true in the future that would tell me that I need to start re-examining my model? If you're not doing that, it's going to be very likely that you are going to be too slow to react to changes in the world.
[00:47:03] MB: Such a great insight. That brings me back to something you touched on earlier and we've started to share some of these themes. I’d be really curious to explore the topic a little bit more of how we start to build a vocabulary, or a shared framework for discussing our thinking, discussing our decision-making and working in conjunction with other people to make better decisions.
[00:47:28] AD: Yeah, that's a really good question. One of the things that I go pretty deeply into in the book is how to actually create good conversations that are going to improve your decision-making. The first thing that I really try to get people to understand is that you have to be meaning the same thing when you speak. It's not just that you have to have a shared vocabulary. It's that you need to know what those words mean. The reason for that and then I’ll give you the example of why that is, but of an example of how that goes wrong, is that really, what we're trying to do when we're making group decisions is to access the different perspectives that are living in the group.
We know the aphorism, two heads are better than one. That's because when we bring two heads together, what we think is happening is that those two people have different perspectives on the situation. It's not just that they may hold different facts in their possession, which would be really good to surface. Even if they were looking at the same facts, they may come to very different conclusions about those facts, because data are not truth. Whenever we get data, we're just modeling that data to try to figure out what we think the appropriate model of the world is and then we're behaving accordingly.
The problem is that that intuition that we have that two heads are better than one, just really doesn't express itself generally when you look at group decision-making. We've all heard of group think and bandwagoning, where people just jump on the bandwagon. We know that groups coalesce around consensus too often. The part of the problem is that there's all sorts of different ways in which we hide disagreement from each other. First of all, just in terms of the science, super, super simple, beautiful, elegant study that shows that groups have a lot of problems in the way that they make decisions, comes from Richard Zeckhauser and Dan Levy over at Harvard. They did the simplest thing. They took a room full of students, a class and they asked the class questions as you do in the class, where you're asking questions and they're taking a poll of the students.
For one group of students, they had the students raise their hand in answer to the question. They put up some question that had some answer, like A, B, or C and they'd have people – how many people think it's A, how many people think it's B, how many people think it's C. What they found was that when you had the hand-raising exercise, in other words, this was happening in public, super majorities formed. Meaning, the majority of the class would settle in on one of the answers.
They then took another group of students and they gave them clickers. These were private to them and they had them answered by clicker. When they answered by clicker, the super majorities completely disappeared. What this tells us is that we have a problem with group discussion, because we end up with this appearance of consensus. It turns out that if you could somehow get into somebody's private mind, you would find out that that consensus didn't really exist. It just appeared to exist.
Let's just start there at that problem. Part of the way to solve this is to create more precision in the way that we talk to each other and a shared sense of how you're supposed to actually walk through a decision in order to surface the different opinions that live in the heads of the people in the group. Now, I can give you an example.
There was a really great survey done by Michael Mauboussin and Andrew Mauboussin. It was from Phil Tetlock, who's written super forecasting, talks a lot about these natural language terms that we use to describe probabilities, things like likely and real possibility and maybe and sometimes and slam dunk and always and never. These are all words that are implying some probability and we use them a lot when we talk to each other.
Like if we were talking about some tactic maybe that we wanted to implement. You might say to me, “Oh, well I think it has a real possibility of succeeding,” and then I nod back to you in agreement. Phil said, that the problem with these words are that first of all, they have very broad meaning, so not everybody means the same thing by them. Second of all, they let you get out of it. When you say, “Well, I mean, it was a real possibility, because there's no precision to it, it doesn't allow you to close any feedback loops.”
Michael Mauboussin and Andrew Mauboussin decided to do a survey. There's 20 natural language terms. The survey is in my book. It's a great thing to do as a team exercise. He just surveyed people and said, when you're thinking about one of these words, what do you think the probability is that something is going to happen in the future for you to use this word? If I say this tactic has a real possibility of working, what does that mean? What's the likelihood that it actually works and speak precisely, please and percentages.
They had people fill out this survey and it turns out that people just don't agree at all on what these terms actually mean when you actually ask them to speak with precision. Surprisingly, people don't agree on what always and never means. That's pretty surprising. Always means somewhere between 95% and a 100% of the time, depending on who you ask. Never, I think might be between zero and 10% of the time. I may have that reversed.
Here is a really surprising one. Real possibility. The biggest range I’ve ever gotten in doing this exercise was 16% to 81%. Think about what that's doing to your group discussions. People are saying things like, I think this is a real possibility and people are nodding in agreement. I haven't ever uncovered that Matt who's nodding at me in agreement, actually thinks that means 20% of the time. What I think is that it means 65% of the time. We go along our merry way believing that we have agreed, because I never actually figured out how are we supposed to talk to each other, in order to find out where we have dispersion.
If you hide the dispersion, you can't have a good group process, because you're not actually getting all that good stuff that comes from having more people involved in the decision, which is really different perspectives and different knowledge.
[00:53:41] AF: I don't know. I think I can tell you that never happens to me.
[00:53:44] AD: Ah, very funny.
[00:53:47] AF: No, but extremely powerful points. I think it's crazy when you ask people to put those percentages behind it, the results you get and just the lack – the realization that we're really not even speaking the same language most of the time.
[00:54:01] AD: No. One of the nice things about this exercise and again, it's in how to decide is that I get teams to do the exercise. Then they find out, “Oh, my gosh. I can't even believe Austin over here thought that meant 25%. What? I thought it was 50%.” I exposed to them that they actually aren't speaking the same language. Then I tell them, well, here's the good news. Everybody on the team now has a list for themselves.
When you feel like you want to say real possibility, please just go refer to your list and say what the percentage is that you have next to real possibility, instead of using that term. Likewise, when we think about a shared understanding of what process is, if I say a real possibility to you, you should say to me, “Well, what do you mean by that? Could you give me a probability?” What this allows us to do is actually, surface where we disagree. Then you have these amazing conversations that occur.
Because I think that part of the problem is that first of all, humans like to agree, for the same reason that when I make a decision, or I have a belief and new information comes in in the future that maybe says that I should change that belief, I don't, because I’m really trying to protect my identity.
I also think that naturally, that when I’m talking to other people, that really what I’m trying to get them to do is certify my beliefs. In other words, I want agreement. We tend to be lingering in the agreement, as opposed to surfacing the disagreement, which is what is actually really interesting. The fact that we all agree that the earth is round is uninteresting. Why should we talk about it? We should actually find out all the places that we disagree. Yet, I’m sure, if you think about meetings that you've had in the past, you immediately realized that it's a lot of, I think that people should be wearing masks and I tell you why.
Then you say, “Oh, I completely agree with you and let me also give my five-minute dissertation on why I think that's correct.” Then Matt says, “Oh, I also agree and let me give my five minutes.” 15 minutes into the meeting, we've all spent a whole bunch of time about something that is non-controversial, at least within our group, that we all happen to agree on. We're not instead talking about well, what do you think about, whether it's safe even with a mask on to go into a grocery store? We may have there, a lot of dispersion around that, but we haven't actually discussed that, because we're lingering on the stuff that makes us feel good, where we all agree and we can all signal to each other that we're part of a group. This is really in a microcosm of the problem with the political identity stuff that's been happening in terms of that driving your beliefs as well.
[00:56:46] AF: Yeah, it's funny. I think that happens a lot, not only in work situations where you're justifying the decisions you've made, or the process you're following. A lot of times when you're hanging out with people in a social setting and something that can be controversial comes up, you all agree. It's like, you spend so much time sitting there, patting each other on the back, basically reaffirming your own decisions, as opposed to maybe seeking points of maybe not intentional controversy, but points of growth.
I mean, I think, I love the analogy of a trench, but in my mind, there's nothing more dangerous than a deep trench and being able to – I always think that the biggest superpower someone can have, especially in their adult life and as they get older is the willingness and the ability to let go of a long hell believe.
[00:57:34] AD: Absolutely. Well, you see here, we're doing it. I couldn't agree more.
[00:57:36] AF: Well, based on new information. I guess, I should clarify. Not just change your belief as you want to.
[00:57:41] AD: Well, yeah. You don't want to randomly change your beliefs, but the open-mindedness to corrective information. I want to linger on that for a second, not because we agree, but I think because there's actually value. If we were to think about the decision process, we can think about at the tail-end of the process, you've got certain options that you're considering. When you choose an option, what you've chosen is the set of possible features that could unfold. For any given option, there's different ways that that could turn out.
The first thing we want to do when we're thinking about options is identify what those possibilities are. Then once we've identified what those possibilities are, we want to think about what the likelihood of those different things are. Very simple. If I go through a red light, I could think about what's the probability I get in an accident, what's the probability I get a ticket, what's the probability I go through fine and nothing happens to me and we could think about what those futures are.
If I’m thinking about hiring a job candidate, I could say, let's say that I’m particularly interested in, concerned about turnover, I could say, I want to think about one outcome would be they leave within a year. Another outcome could be they leave between a year and two years and another outcome could be they stay beyond that and then I could think about a probability for each of those possibilities.
What's really important to think about is that when you choose an option, there's all these different ways that it could turn out, but what tells you which way you observe on that particular time is due to luck. If there's some outcome that's going to occur 5% of the time, what that means just definitely, it's axiomatic is that it's going to happen 5% of the time. The thing is that you don't know which 5% you're going to observe, when you're going to see that 5%.
On any given time that you make that decision that has say, a bad outcome 5% of the time, you could get a bad outcome on that time and you don't have any control over it. That's the influence of luck.
The way I think about luck is luck is something that you need to see clearly, but you can't actually do much about. You need to see it clearly, because you need to actually know what the possibilities are and what the probabilities are as close as possible, in order to make a good decision. I’m actually interested in this particular case and what's happening prior to that.
Essentially, what's happening is that our beliefs are informing everything about the decision. Our beliefs inform what our goals are, what our values are, beliefs inform what we think our options are. Our beliefs inform what resources we believe we have to put toward those options. Frankly, it's our beliefs that tell us what we think the possible outcomes are and what we think the probability of those outcomes, of each of those outcomes is. That's all driven by our beliefs.
Here's the problem. We can think about, this is where we get the intervention of imperfect information, because when we think about what we know, versus what we don't know, it's like, what we know could fit on the head of a pin and what we don't know is the size of the universe. Now, the issue is that our beliefs, really if you think about it are the foundation of all of our decisions, because they're informing everything that come everything about the decision.
Our decision-making house is built on this foundation and that foundation has two problems with it. One is that there are inaccuracies in that foundation. Some of the things that we believe, in fact, many of them are not really perfectly true. They tend not to be perfectly false either. They're generally somewhere in between, but we can think about those as cracks in the foundation.
Then the other problem that I just said is that the foundation is flimsy. What we know compared to the knowable – what there is to know about the world is quite small, and so what we'd really like to be doing is broadening that foundation as well, to beef it up.
Our decision-making house is just sitting on this weak foundation. Really if you think about it, the biggest thing that you want to do as a decision-maker is to improve that foundation. In other words, improve the quality and the breadth of your beliefs. The only way you can do that is to go search around in the universe of stuff that you don't know and collide with both new information and corrective information.
Now, some of that you could Google, but much of that information is going to live in other people's heads. It becomes imperative to be interacting with people who may hold different views than you, A, and B, interact with them in a way where they would be willing to express those views to you. Then you have to remain open-minded to them. Because when we think about things like, confirmation bias for example, where we're really interacting with the world in a way that's seeking out confirmation of the things we already believe, that's a walking through the universe of stuff we don't know problem, which is that we're not taking a random walk through it. We're specifically walking into the little sections of it and looking around in the parts that already agree with us.
How on earth is that going to help the foundation of your decisions? It's going to reinforce the inaccuracies in it and it's certainly not going to broaden your knowledge, because they're repeating back to you things that you already believe. I think what's interesting about it is that if I say to people, would you like to become a better decision-maker over time? They'll say yes. I’ve yet to meet somebody who says no to that.
I say, “Well, okay. Let me just ask you hypothetically, in order to improve your decision-making over time, could you imagine that you may have to find out that some decisions that you've made in the past have been poor?” They say yes. I say, “Could you imagine that you need to learn a bunch of new stuff?” Yes. “Could you imagine that you're going to have to find out that some belief that you hold pretty dearly maybe you need to recalibrate in some way?” They'll say yes. They'll say that all in theory, they'll recognize that that's true.
In practice, when they're actually interacting with the information in the moment that they collide with that stuff that's corrective, they either don't see it. They don't seek it out, or they reject it altogether and they pull, as I say, pull it down into the trench and just say, “Well, actually, I don't think that that's what that means at all.” Then they'll do some interesting narrative spinning and there you go.
This idea of how are you thinking about how do I change my mind, how do I find that information that's going to help me reverse my beliefs, this literally it's the key. If you don't do that, you can't improve your decision-making. It's like, everything that you need to be doing has to be directed around that.
[01:04:17] AF: I want to tell you about this amazing online community that I’m a part of. It's the ultimate Knowledge Hub from The Hustle called Trends. You can learn all you need to know about Trends in virtually any space from business, personal development, psychology and more. It's an incredibly powerful tool and helps you identify trends before they happen, so you can be the first to capitalize on new and emerging trends in any space. Trends will give you the network and information you need to succeed.
Trends gives me access to a community of industry leaders in virtually every field, where I learn how to capitalize on emerging opportunities. Trends is a place where I workshop ideas and I network with other entrepreneurs and investors. One thing I love to do is attend their weekly live lectures. They have experts who teach growth strategies, SEO or my favorit,e how to send the perfect cold e-mail. A lot of our sponsors know, I do send quite a bit of cold e-mails.
Trends has exclusive research with intriguing topics to help inspire, like the 30 companies defining the future of media and pop culture, or data on thousands of successful Kickstarter projects, for example. I’ve always been fascinated by supplements and sleep. Lately, I’ve been discussing the new trend of sleep supplements and how they can help us get to sleep faster and get more out of our time asleep. I’m enjoying Trends so much that I want to share it with you.
Right now, you can get your first two weeks of Trends for just $1. Go to trends.co. That's C-O. Trends.co/success to start your $1 two-week trial. That's trends.co/success for your $1 two-week trial. Trends.co/success.
[01:06:08] MB: Yeah. I mean, that is such a critical foundational element of all of this. You did such an incredible job just bringing this whole conversation together into really, the synthesis of how you can think about decision-making, how you can think about investigating your own biases, how to start to become someone who's trying to improve their thought process, instead of just constantly feel validated and feel they're right and not really have the courage to investigate their own thinking.
Annie, for somebody who wants to start putting this into practice in some way, what is one simple action item, or starting point that they can execute on to take a leap into this broader journey?
[01:06:59] AD: I would say, yeah, so the single most important thing that I think that people can do is start figuring out how to actually elicit other people's true opinions. Let me talk about it in a one-on-one basis and then how you might actually implement this into a team setting. Generally, the way that people elicit feedback goes awry. The reason it goes awry is that usually, when I offer you my opinion first about the thing that I’m trying to get feedback, what I’ve done is put you in a position where if you have a different opinion, you have to actually publicly disagree with me.
Let me explain what I mean. If I was thinking about a poker hand, for example, the way that most people ask about poker hands is this person raised in front of me and I’ll tell you some facts about that person and the situation. I’ll say, this person raised in front of me and then I looked down at ace, queen and I raised, what do you think I should have done? Now the problem is that the feedback I’m trying to elicit from you, which is what should my action have been, I have just now told you what I actually did.
If you think that I shouldn't have raised, I’ve actually put you in a bad position, at least as it comes to these cognitive biases, because now you have to disagree with me if you actually do disagree with me. What's going to happen there is generally, that either you're going to continue to disagree with me, but then not actually say so, because maybe you don't want to just embarrass me, or maybe you don't want to embarrass yourself, because you think I’m more of an expert than you are. That could be one thing that happens.
Another is that while I’m speaking and I say, so I raised, what do you think I should do? Your opinion could shift without you knowing it. This is particularly if I’m in a leadership position, or if I’m perceived as a subject matter expert. Or if I’m for some reason, you feel I have higher status than you do. Your opinion may bend toward mine. That is another thing that could happen.
The third thing is that if you do disagree with me, you'll tend to put it in bubble wrap. Yeah. No, I think raising is pretty good there, but did you think about this other thing, as an example. We can think about this as we're thinking about eliciting feedback on anything. I think Forrest Gump was a horrible movie. What do you think? We're on a hiring committee and we're looking at two candidates. I’m like, “I think Susan’s a superstar. What do you think?” All these ways, we offer our own opinion first. What we're doing is infecting the other person with our beliefs.
The simplest thing that you could do, the biggest change you could make in your life is when you're trying to get somebody else's opinion, don't offer them yours first. What I want to do is say to you, the person raised in front of me, I had ace, queen, what do you think I should do? If you say to me, “Well, what did you do?” Which is normally the question that you get back. “Hey, we just interviewed those two candidates, which one do you like better?” People will say, “Well, what do you think of them?”
They're doing that because they want to make sure you're on the same page, but you're specifically trying to find out if you're not. By doing this, I allow you to express your true opinion. Then now, I can actually find out, “Oh, actually, Matt thinks I should have just called there, so let me explore that. Let me find out why Matt thinks that before I offer you any of my own thoughts. This is going to help me to collide with more of that universe that I want to be interacting with in terms of knowledge.”
How do you actually implement that into a team? That becomes harder, because if Austin, Matt and Annie are having a conversation and I do this in the way that I speak to Matt, now when Matt expresses his opinion, obviously he's now infected Austin. I want to avoid that. This is now the second biggest thing that you could do in your life is think about the feedback that you're trying to get from your team prior to going in and having a meeting on the team and make sure that you elicit it from each team member independently.
In other words, in a way that the other team members cannot see initially. As an example, if you're just thinking, if you're on a hiring committee and four people have just interviewed the two candidates, figure out what are the things that matter to you in terms of the candidate, so it may be of the candidates we're going to see, where does this candidate sit on a scale of zero to 10 compared to that reference group?
Or you might say on a scale of zero to five, how autonomous do I think this individual will be? We can think about what those are. Another thing that we could do is we could be doing a pre-mortem, which is just thinking about the ways things can go wrong in the future and I could do this as the feedback that I’m trying to elicit. In any case, I’m thinking about what that feedback is. Then what I do is we all interview the two candidates and then we literally don't talk. I send this form out, everybody replies to the form independently. I collate all of that together and then I reveal it to the group before we get into the meeting and what we find out is that in the universe of candidates that we're going to see, Annie thinks the person was an eight, Matt thinks they were a six and maybe Austin thinks they were a five.
Now we find out oh, we actually have a pretty big spread on this. Now this is what we should discuss. What you're going to find out when you do that pre-work is a couple of great things come out of it. The first is you're going to find out there's more dispersion than you think, which is a good thing. The second thing is you're going to find out that there's areas of agreement. Also, a good thing. When you go into the meeting now, everybody's going to know what you need to talk about, which is we all agree on this. Let's actually talk about the areas of dispersion.
Now people can give their rationales, not in order to convince other people of their point of view, but just to express them about why they think that all these different things are true. In that process, not only do you collide with more of these different perspectives, but in having people actually talk about why they believe these things that are different than other people in the room, the whole group becomes more educated. They learn more facts. They get access to different ways of thinking about problems. They may have access to different data somebody else could be bringing different data to bear than you did on the problem and so on and so forth.
If that's all that you did was change that thing about the way that you interact with other people, you would be so much better off, because you would be constantly be learning and growing from other people, which is really what you're trying to do.
[01:13:34] MB: Well, Annie. I know we could go on and talk about decision-making and all of these strategies for hours and hours, but this has been an incredible conversation. Such, such great insights. So many powerful and applicable ways to improve our own decision-making. For listeners who want to find the book, who want to find out more about you and your work, what is the best place for them to do that online?
[01:13:56] AD: Oh, so you can go to annieduke.com. There's a contact form there, so you can e-mail me there. There's links if you want to hire me for stuff. There's a newsletter link there. Obviously, I do consulting and speaking and whatnot, but there's also a newsletter link there. You can find out how to buy the books there, so that how to decide and Thinking In Bets. You can also follow me on Twitter. I’m actually pretty active on Twitter and that's @AnnieDuke. I would say, those are probably the two best ways to get in touch with me. You can buy my books on the usual suspects.
[01:14:27] MB: Well, Annie. Thank you so much for coming on the show, for sharing all of this wisdom. It was great to have you back on here, an even better conversation where we really got into tons of great insights about better decision-making.
[01:14:39] AD: Thank you. Thank you for having me.
[01:14:41] MB: Thank you so much for listening to the Science of Success. We created this show to help you our listeners, master evidence-based growth. I love hearing from listeners. If you want to reach out, share your story, or just say hi, shoot me an e-mail. My e-mail is matt@successpodcast.com. That’s M-A-T-T@successpodcast.com. I’d love to hear from you and I read and respond to every single listener e-mail.
I’m going to give you three reasons why you should sign up for our e-mail list today by going to successpodcast.com, signing up right on the homepage. There are some incredible stuff that’s only available to those on the e-mail list, so be sure to sign up, including an exclusive curated weekly e-mail from us called Mindset Monday, which is short, simple, filled with articles, stories, things that we found interesting and fascinating in the world of evidence-based growth in the last week.
Next, you're going to get an exclusive chance to shape the show, including voting on guests, submitting your own personal questions that we’ll ask guests on air and much more. Lastly, you’re going to get a free guide we created based on listener demand, our most popular guide, which is called How to Organize and Remember Everything, you can get it completely for free along with another surprise bonus guide by signing up and joining the e-mail list today. Again, you can do that at successpodcast.com, sign up right at the homepage, or if you're on the go, just text the word SMARTER, S-M-A-R-T-E-R to the number 44-222.
Remember, the greatest compliment you can give us is a referral to a friend either live or online. If you enjoyed this episode, please leave us an awesome review and subscribe on iTunes, because that helps boost the algorithm, that helps us move up the iTunes rankings and helps more people discover the Science of Success.
Don't forget, if you want to get all the incredible information we talk about in the show, links transcripts, everything we discuss and much more, be sure to check out our show notes. You can get those at successpodcast.com, just hit the show notes button right at the top.
Thanks again, and we'll see you on the next episode of the Science of Success.