Is optimism rational?

Q: What do you mean when you talk about optimism?
We have a very precise definition: a positive bias relative to some objective criteria. And, more specifically, a bias in the preferred direction of the likelihood of an event relative to an objective measure of the likelihood of that event.

For example, if you're rolling the dice and need a one, the likelihood of that is one-sixth, right? You're optimistically biased about it if you believe you're likely to roll a one — or even if you believe there's a 25% chance you'll get that one.

Q: Is optimism irrational?
It's mistaken. It's a bias. Whether it's irrational depends on what the consequences are of that bias. And that's a huge debate in the literature, and has been for some time. Presumably it's irrational if not only is it mistaken, but also the net consequences of it are negative. Then I'm comfortable calling it irrational. However, there may be positive consequences to being positively biased.

There is a fairly widely held belief among some researchers that some optimism is good. It's a little bit like red wine: In moderate amounts, it's good for you.

There are a number of reasons why it might be good to be optimistically biased. For example, if people are by nature a little too risk-averse, a little optimism helps counterbalance that, gets them to take a few risks they might not otherwise.

Another reason is that there is, in economic-speak, positive utility from having positive expectations, in and of itself. Setting aside any decision making that you do based on the expectations, just consider how it feels to walk through life believing that your favorite football team is going to win this weekend, that your relationship is going to work out, that your financial investments are going to pay off. If you're thinking things are generally good, you're going to be happier; if you're thinking things are bad, you're going to be unhappy.

Now, that has to be offset against what happens once these things are realized: after the game is over or the investment is complete or the relationship is over. If you have been positively biased all along, then you're going to be more disappointed in the end than you would have been if you hadn't been positively biased.

Now, are those two things equal? If those two things are equal, then there's no net positive to optimism. But here's something we know to be true: people adapt to bad news much more readily than they expect they will. So it could well be the case that those things are asymmetric. That there's more utility to be gained or lost ahead of an event than after an event. The disappointment isn't as big a deal as we think it is. And if that's the case, it's nice to have a little optimism because it feels better.

A third reason is that there can be a self-fulfilling nature to positive expectations. If you have positive expectations for somebody, they actually perform better. And if that's the way the world works, a little positive bias is a good thing.

Q: You have a paper called "Prescribed Optimism" that is about, essentially, how optimistic people think one should be. Does that shed light on this question of whether optimism is good or bad?
It's at least suggestive that optimism is a good thing.

In that paper we ask people about a variety of scenarios. They're intended to be real-world settings: one was a dinner party, one was applying for a scholarship, one was preparing for an exam. And then we had different conditions in which we varied how much control the person had over the situation in the scenario.

Then we asked people, in each of those scenarios, how optimistic they thought they would be, how optimistic they thought other people would be, and how optimistic they thought other people should be.

We were surprised to find that across all the scenarios and across all the conditions, people were optimistic. They also reported that others were optimistic. And most provocatively, they told us that they thought that not only were people optimistically biased, but that they should be even more optimistically biased. And we think that is suggestive that there may be something adaptive — rational — about optimism.

Q: You have a paper about how optimistic MBA students are about their grades, and how that changes over the course of a year. What is that paper trying to get at?
That paper is part of a broader enterprise, which is understanding what happens to optimism with experience. Maybe people are optimistic when they first walk in the door, but what happens when they get feedback? Do they learn?

It's a great place to explore this intersection between psychology and economics. Economics leans heavily on rational behavior and learning with experience, whereas psychology identifies the bias. So let's put them together and find out what happens.

This study is about whether MBA students learn how good they are in the classroom over the course of their first year in the MBA program. We followed a cohort of MBAs at another university. As soon as they came into school, we asked them how well they thought they would do in the classroom. And then every quarter we came back and asked them again, how well do you think you're going to do this term? Meanwhile we're observing their actual grades.

And what we found was that at the beginning of the year, just like all the psych studies show, they were optimistic about how well they were going to do. But we also found that at the end of the first year, after four terms, they are still as biased as they were at the beginning of the year. It's one thing to be biased when you're never been in an MBA classroom before. But to have the same bias at the end of the year is a different matter.

However, they are better calibrated at the end of the year. That is, there's a greater relationship between those who say they're going to do well and how they do, and those who say they're going to do poorly and how they do. So they are paying attention. There's a form of learning there. But they're still biased.

So that's a starting point for us. Here's a domain where they care a lot, they get real feedback, there's some consequences to these beliefs, and yet we see optimism persist.

We have a related project that allows us to go into a little more detail on what's going on. For the past two football seasons, my colleague Joe Simmons, across the hall, and another colleague, David Armor, and I have studied football fans' predictions about their teams.

Before the season starts, we survey them and find out all kinds of things about them — how much they like different teams, how much they know about different teams, some information about themselves. But the most important thing we learn is who their favorite team is. And then we can look at their predictions about their favorite team versus other people's predictions about that team. Then we ask them every week of the season to predict what's going to happen in that week's games. And we look at whether they are biased and how that bias changes over time.

What we find is that in the first week, there's a four-point difference between how a fan thinks his favorite team is going to do relative to how everyone else thinks that team is going to do. Which is a big bias. With a little experience, in the first couple of weeks of the season, that bias does come down some. But then it flattens out and persists through the entire rest of the year. So at the end of 17 weeks of feedback, they are as biased as they were after four weeks.

Now, with the MBA grade study, we're looking at something that people have control over, and there might be a reason to have a little optimistic bias. Maybe it provides a goal for them, maybe it reflects on their self-esteem. Maybe they want to present themselves a certain way to the researcher, even if the researcher is anonymous.

We wanted to get away from some of those things, so we went to football, where the fans don't have any control. Now, there are other downsides to that. There's not as much consequence here, for example. But at this point we have one study where they have no control and there's less consequence and another where they have more consequence but also more control. And we find the same pattern in both situations. It seems to be pretty robust.

Q: What's next in the larger project looking at how optimism responds to experience?
We're observing this pattern where people get better calibrated over time but the bias persists. That's a very distinct model: it says they don't have their head completely in the sand, but they've got this baked-in optimistic bias. And one challenge for us is to step away from these studies and come up with a general learning model when you have strong preferences. What would a model look like that generates that pattern? And so we're considering a few different possibilities.

A model that would generate our pattern is that they do pay attention to the evidence, and they do pay attention equally to positive and negative, but in all cases they interpret it more positively than it actually is. So they get more calibrated over time, because they are incorporating good news and bad news. But the bias remains because they're always taking the best possible version of everything they hear.

Another is that people are forever thinking it's a new day. If I believe today is different — today I'm trying harder, I've got a new boss, weather's different — I can ignore older feedback. That would give you the same pattern. You are paying some attention to the data, so you're getting a little better calibrated, but the bias persists because you're forever ignoring older data.

My colleague Erica Dawson at SOM and I are playing around with this and working on studies trying to find out whether this is part of it. The enterprise is to try to create a general model that would apply to situations where people are motivated, like health, relationships, and investments.

Interview conducted and edited by Ben Mattison