In this episode of the Inside the Strategy Room podcast, partner Tim Koller and Dan Lovallo, a professor of business strategy at the University of Sydney, and Sean Brown discuss how executives can use a set of reference cases to quickly build an “outside view” that will help debias their forecasts. They also talk through who is best placed to deploy this debiasing technique and the outside view can become part of your ongoing strategy process. You can listen to the episode on Apple Podcasts, Spotify, or Google Podcasts.
Podcast transcript
Sean Brown: From McKinsey & Company’s Strategy and Corporate Finance Practice, I’m Sean Brown. Welcome to Inside the Strategy Room. Today we’re talking about bias. Specifically, how executives can fall prey to cognitive and organizational biases that get in the way of good decision making. McKinsey partner Tim Koller and Dan Lovallo, a former McKinsey consultant and a professor of business strategy at the University of Sydney, recently wrote a series of articles on this topic. Today’s discussion with be about how executives can take what is called the “outside view” to remove biases. Tim and Dan joined me recently in our New York office to discuss the article.
Sean Brown: Tim, let’s start with an overview of the Bias Busters article series. What prompted this series, and what are you trying to achieve?
Tim Koller: Thank you, Sean. Over several years we’ve observed that our clients often run into difficulties making important strategic decisions about allocating resources, investments, and those kinds of things. Often it is because there are internal biases in the way they make decisions. Their decision-making processes aren’t able to overcome a lot of the biases that we all have as humans. Talking to researchers and others, we’ve discovered that in order to overcome those biases you need to overcome the way the organizations make decisions, the rules that they use, and the procedures that they go through. We’ve been working on helping companies overcome those biases. Now we have a whole collection of biases and how to overcome them. Dan came up with a great idea and said, “Let’s write a series of very short articles to make each one of them very accessible and focus in particular on how you overcome these biases by changing the way organizational decisions are made.”
Subscribe to the Inside the Strategy Room podcast
Sean Brown: Dan, in this particular article, which is the first in the series, you discuss this notion of the “outside view” versus the “inside view.” Could you tell us a bit more about what that means?
Dan Lovallo: Sean, quite a while ago, I think about 1993, Daniel Kahneman and I came up with the terms the “inside view” and the “outside view” to explain different modes of thinking. And, as an aside, I’ve been carrying him since then. The inside view is the way people usually think about decisions. In other words, they start on the problem and they build the case. In business, the case often starts with an Excel spreadsheet. You start putting in numbers, and they look into a crystal ball and try to see the future and plan that future out in advance. And that’s the natural way to think.
Another way to do it is to use lots of analogies or cases. When you use these cases, you take a more statistical view. These cases have been used widely now in the UK. For transportation projects, you are required to use a reference class. In Iraq, Colonel Kalev Sepp at the nadir of the Iraq War, came up with 53 cases in 72 hours to help renavigate where that conflict was going, with temporary success. The outside view is statistical. That’s the basic difference between the inside view and the outside view.
Sean Brown: Taking the outside view sounds like a really good idea. How do people actually do it?
Dan Lovallo: The outside view combats a number of biases, including optimism, overconfidence, and even anchoring. Let’s focus on optimism and forecast. That’s the main thing the outside view attacks. There are several ways you can do it. Tim, could you talk about the momentum case, which is an outside-view technique that works quite well?
Tim Koller: One of the things we are working with clients on, particularly when they are developing their strategies or their forecasts for performance, is helping them to think from the perspective of how an outsider might look at their markets, how they’re evolving, and therefore whether the plans are realistic or not.
What we’ve found in a number of high-profile companies is they set targets for performance for the year, let’s say, that are divorced from what’s really going on in their marketplace. Because they’re always optimistic that they’re going to be able to overcome whatever forces are out there. And, of course, they’re not taking into consideration what’s going on with the competition, what’s going on with the bigger market.
Dan Lovallo: And that’s the inside view. You focus on what you’re going to do, and you don’t balance it against an outside view. In this case, the momentum case.
Tim Koller: For example, companies say, “We’re going to cut costs. And it’s going to increase our profits by this much,” without realizing their competitors are cutting costs as well. They’re probably going to end up passing those savings on.
Dan Lovallo: That’s another manifestation of the inside view. People act like they’re playing up against a brick wall rather than a partner. It’s called competition neglect. The number of companies that set a new market-entry plan and don’t plan for the reaction of the competitors is about 90 percent.
Sean Brown: It sounds like the outside view can provide a counterweight to that internal optimism. In your experience, how far apart are those two views? Are we talking massive differences between the outside view and the inside view?
Dan Lovallo: I can tell a story about the movie industry. I thought I was going to get rich. That’s the important part of this story. I did a study, along with Colin Camerer, who is a good friend, and recently won the MacArthur grant. What we were able to do was forecast movies out of the domestic box office based simply on a poster and a one- or two-line description on Metacritic. What we got was the similarity ratings between the focal movie we were trying to predict and a reference class of 40 other movies.
Sean Brown: Superhero movies, for example, or whatever it may be?
Dan Lovallo: Right. The reference class was formed by genre, actors, and story line. That’s it. We took the intersection of those, and there were more than enough movies. When we added similarity ratings to the typical regression things they use like budget and whether it’s an action movie or not, our mean average residual error was 25 percent, which is exceedingly low for something like that based on all they needed was a poster and a paragraph about the movie. And the cost of doing this is just giving the subjects a free movie ticket.
The reason I thought I was going to get rich was because I got the meeting with the head of the studio, and I walked him through our data. I said, “What’s your error on average?” And he said, “One hundred percent. We think a movie is going make $100 million and it makes $50 million, or it makes $150 million. We’re just not close.” And I said, “Well, you know, I can cut that by 75 percent. Let’s work together.” And he thought for a while, and said, “No, I don’t think I can do it.” And I said, “Well, how many analogies do you use to make forecasts of movies at the beginning?” And he said, “Well, sometimes we use one.” In other words, they use almost all the inside view.
And I said, “What’s the most you’ve ever used?” And he said, “Two.” I said, “Well, do you believe what I’m doing? If you don’t believe me that we did this beforehand, give me your next slate of movies. If I’m helpful, you pay me, if not, it’s free.” I thought I can’t go wrong there. And he said, and this is where he was very honest, he said, “No, it’s not that. I get to pick 12 movies a year. And I’m only in this job for a few years. I don’t want evidence out there that I should’ve done something different.” Very candid. And there was no money for me.
Sean Brown: Well, hopefully there’s some money for our listeners. Tim, it sounded like you were going to ask a question?
Would you like to learn more about our Strategy & Corporate Finance Practice?
Tim Koller: I was just going to ask Dan, in my experience, it’s very difficult for the people who are directly responsible to do the outside view. Is it easier to do the outside view if you have someone else in the organization challenging the inside view, as opposed to the person, or the business-unit head, or the project lead the doing the outside view?
Dan Lovallo: Well, the best is when the government mandates it, as in the case of the UK infrastructure. I think you need to make it a part of how you forecast. It has to become part of your process. Somebody needs to champion it once. But people think it’s really hard to do, because you’ve got to collect all these comparable cases, which in the case of movies was easy. But they’re still using one analogy. But I think Colonel Kalev Sepp put the lie to that when he came up with 53 examples of counterinsurgencies. He didn’t go back hundreds of years. He just went back, I’m not quite sure, I think it was a back to World War I or maybe World War II. And he did that in 48 hours. If you want to do it, you can do it.
Tim Koller: One of the things I’ve observed in a lot of larger organizations is that there’s no mechanism for creating the outside view. And there’s no people to do it. For example, you would think that in most cases, in a large company with multiple business units, that the corporate financial-planning and analysis group would be responsible for evaluating and thinking about the plans that are coming in from the business units. And yet, in many cases, the corporate teams have barely enough time just to add up all the numbers from all the different units, so often you don’t have any systematic outside view. The only information you’re getting is from the business units themselves, from the project leaders, as opposed to someone else looking at it.
One of the advantages private-equity firms have is a team of analysts who is constantly scrutinizing what’s going on in their portfolio companies. Big corporations often don’t have that. In order to have an outside view, and it’d be interesting to get your perspective on it, Dan, I also think you need to have a mechanism to make sure it’s not just the project leader doing it, or the business-unit leader doing it, but there’s also someone there to challenge it.
Sean Brown: Like a disinterested party, if you will?
Dan Lovallo: Danny and I really screwed up when we named this the outside view. And we only came to realize it later. Because we don’t mean an outside party when we’re talking about the outside view. We mean statistical data. And that often naturally gets confused. We didn’t think of it because we couldn’t foresee how it would be used.
Tim Koller: In larger organizations sometimes, it’s easier for someone outside the business unit or the project to do that statistical analysis or take that more rigorous scientific approach, as opposed to the people directly responsible for that project.
Dan Lovallo: What you’re saying is true. That makes sense to me. But ideally, I would want them to be in the same organization. It should be part of the due diligence for every important forecast.
Sean Brown: It seems that one of the key challenges is figuring out what those reference cases are. Are there any specific skill sets or techniques that help somebody who’s trying to take the outside view to do it effectively?
Dan Lovallo: It’s part art and part science. A way to think about it, for those who know how to do regressions, and I’m not going to assume everyone does, but they would be the main variables in your regression. In the case of movies, you know, it’s genre, actors, story line, and things like that. You could also thin it by budget. But budget is a continuous variable, so you might as well just plug it into the regression. You can do it with almost anything. For example, EMI with the CT scanner. If you wanted to make a reference class for how they were going to do on entry, you might look at new technology and advancements in medical devices. That would be one thing. And de novo entrants in the field, because they weren’t in the space. That forms your reference class and gives you a likelihood of success. In their case, the likelihood of success was quite low, and they ended up losing money on the project in the end, even though the scientist who invented the CT scanner, Godfrey Hounsfield, won the Nobel Prize. There was a silver lining, and it saved a lot of lives. But big companies like Siemens came in, and they had all the technology. Usually the winners are people who are already in the space and then they see the work that the new technologies produce, and they come and say thank you.
Tim Koller: Just another example of that in the corporate world. I recently read an article about autonomous vehicles and the impact electrification would have on them. The supposed expert was suggesting that as a result of the introduction of autonomous vehicles, and different ways of providing services to customers, that car companies would go from making $1,000 to $2,000 of profit per car to $30,000 of profit per car. And, of course, without any backup. The logical approach to looking at the outside view is to look at all the innovations that have occurred over the last 40 years in the automobile industry, and how that affected profits, and how that even affected the profitability of the first innovators to see whether or not in fact you can keep those profits, because everyone copies everyone else so fast. But by looking at what happened in other situations you might be more cautious about that. Does that make sense as an example?
Dan Lovallo: Yeah, absolutely.
Bias busters: Taking the ‘outside view’
Sean Brown: We talked about how a company can do this. Typically, they’ve got to have somebody on the inside who is thinking about this, who understands what the appropriate reference cases are. Dan, you talked about the way that the UK has required this for transportation projects. Can you talk a little bit more about how they actually mandate it? Because I’d imagine if you were running a transportation project and you select specific reference cases, you can tweak the results. Is every bidder on a project required to use the same reference cases, or how does it work?
Dan Lovallo: Your point about the reference cases is everyone should agree on them because you can tweak things. Before you show the answer, you should get buy-in from the team on what the reference cases are so you don’t adjust your reference cases after the fact. I think that’s a very good point. Teams should get buy-in from the entire team including the topmost decision maker on the problem about the reference cases, and then do the analysis. Teams should not show up with a piece of analysis that they may or may not like, and then be told to go back and redo it.
Sean Brown: I’ve read the outside view can also be helpful if you’re about to embark on a major home renovation where people tend to be very optimistic about the time and cost it’ll take to get it done. Have you had any experience with that, Dan?
Dan Lovallo: I’m having some direct experience with choosing the right reference class for my—this is a little morose, but my mother and father passed recently. And my sister and I, for reasons unknown, I’m essentially the seller and she’s the buyer. And we’ve had only one analogy. We’ve agreed that we’re going to agree on the set of reference cases to determine what check gets written to me. It can come in for many things, like home renovations, where you’d look at what goes on in the area. And apparently for probate as well.
Tim Koller: We recently renovated our master bathroom. Without doing this very scientifically, I just automatically assumed that it would take twice as long as the contractor said and cost 30 percent more. I built that in just based on listening to what other people’s experiences were. I did it intuitively without doing it very scientifically.
Sean Brown: And were you pretty much on the money?
Tim Koller: Yeah, pretty much.
Dan Lovallo: Now, an important thing that we shouldn’t finish without saying is, even building the reference class helps you not only with forecast, it helps you generate ideas. Because you’re looking at all these cases, instead of just taking the inside view, or just looking at one analogy or two analogies. You’ve got all this information. And it can give you ideas on different strategy paths to take as well. Collecting the reference classes not only helps forecast, but it helps you be more creative in the strategy room.
Sean Brown: For the reference cases, you don’t want to just look at the outcome. You want to look at what they did too.
Dan Lovallo: Yeah. If you’re getting the information anyway, why not get a little bit more? And it can often give people ideas to do things that they wouldn’t have otherwise done.
Sean Brown: Practically speaking, for companies that are looking at embarking on major capital investments, is this the kind of information that one could pull from financial statements? Or how does one actually dig into this? If you’re in financial planning and analysis, and you’d like to bring the outside view into your evaluation of the next three projects that come before you, how do you do it?
Tim Koller: A lot of it will depend on the industry. It’s not something you can pull from the financial statements. But if, for example, you’re in the chemical industry, firstly, you’ve done these things before yourself, and, secondly, in the chemical industry it’s pretty easy to see what your peers have done, and what’s worked, and what hasn’t worked. Often these are big, visible projects. It’s combination of past projects, with projects inside the company, plus knowledge of what your peers have done. There will be people who will know what’s happened in other companies, and what their experiences were that you can bring into this as well.
Dan Lovallo: This is an important point. There are three types of learning that can go on. One, if you’re the decision maker, you learn from your own past experience. Two, and this is in decreasing order of what gets used, you learn from your own past experience, then you can learn from the past experience of the company or organization that you’re working for. And then the step that people rarely take is learning from others’ experience.
There’s an awful lot of information there. And I haven’t been presented with a problem where I couldn’t come up with a reference. Sometimes it might not be that huge. But part of the theory of the reference class is biblical. It comes from Ecclesiastes. There’s nothing new under the sun. Depending on the lens you’d look at it from, there’s something to be learned, even if you think what you’re doing is completely brand new. If you abstract a little bit, you’ll be able to find a reference.
Sean Brown: I think it’s interesting that you brought up that point that the outside view can actually be based on inside experience. It’s just that you have to be objective about it. This has been great. Any final thoughts you’d like to share before we close out our podcast?
Dan Lovallo: I think when your last statement was about the Bible, you’ve got to stop there.
Sean Brown: You’ve got to drop the mic. Dan, Tim, thank you so much for joining us today. We look forward to our next podcast on bias busters.
Dan Lovallo: Thank you.
Tim Koller: Thank you, Sean.
