Taking a Disciplined Look at Irrational Investors

Q: What questions have guided your career?

At the simplest level, behavioral finance is trying to develop a more psychologically realistic description of how investors think. We can break that down along two dimensions. First, how do people form beliefs about the future? Then, once people have beliefs, how do they make decisions?

Since the beginning of modern finance research in the 1950s, we thought of financial markets as fully rational. In the 1980s, some people started to argue, “We need to take into account the possibility that people aren’t fully rational.” Robert Shiller at Yale and Richard Thaler at the University of Chicago were the most prominent of those voices. Now, both are Nobel Prize winners, but initially they didn’t get much traction. In fact, there was a fairly hostile reception to their views.

So hostile that, when I did my PhD in financial economics at Harvard from 1991 to 1996, we just weren’t taught anything about behavioral finance. I wrote my dissertation in rational finance, but just as I was graduating one of my advisors, Andrei Shleifer, told me about work he had started doing in behavioral finance. It sounded very interesting because it combined mathematics, economics, and psychology. In my heart, I felt like it was right. I guess those conversations changed my life. The hope is that more psychologically realistic models will capture more of the truth about financial markets than the rational paradigm. It struck me as an exciting and meaningful mission, to try to build up this new field, behavioral finance, in a scientific, rigorous way. That’s what I’ve been doing for the past 25 years.

Q: Isn’t it self-evident that people aren’t always rational?

People outside academia may consider it obvious that humans are sometimes irrational and of course that will impact the economy and financial markets. They’re probably right. But it’s a very different thing to actually prove it as a scientific matter.

That’s particularly hard in economics and finance because we can run relatively few controlled experiments, so things move slowly. That’s why the rational paradigm has held on as long as it has.

Q: How did the rational framework come to be so dominant?

It is a natural place to start. Economists can define what rational means with some precision. If you depart from a fully rational model, it’s like walking into a thick jungle. There are dozens of ways people depart from full rationality. It seems impossible to know which to follow.

I look at it differently. Yes, it’s difficult, but we can’t keep working with a model that may be wrong just because the alternative model is difficult. We have to do our best. We have to proceed scientifically.

Q: Why has there been such hostility?

I suspect some of it was the emotion that accompanies major paradigm shifts. If you’ve spent your career working on one model, and someone comes along and tells you it’s wrong and needs to be thrown out, well, that’s threatening. It opens up questions about the meaning of your life’s work.

At the same time, there are genuine scientific disagreements and real intellectual issues to be debated. Sometimes the prices in the stock market rise very high relative to underlying fundamentals. A rational view of that might just be, “Oh, people have just become less risk averse,” while the behavioral finance view might be, “People have overreacted. They’ve gotten too excited, so prices are disconnected from the fundamentals.” Proving one view over another isn’t simple.

There were also methodological concerns about behavioral finance and behavioral economics. The worry was that for any question that came along, we would just flip through a psychology textbook until we found a psychological bias that would seem to resolve the puzzle and let us declare victory.

The natural progression as one paradigm challenges another includes a lot of roadblocks along the way. I believe the methods of behavioral finance are proving their validity. We’ve proceeded by carefully picking out the most robust pieces of psychology, then building models around them. We use those models to make new predictions. Then we test the predictions by bringing more data to bear.

The behavioral frameworks make sense of the world in simple ways that are increasingly supported by the evidence. That’s winning over more and more adherents. Where some fields generate a lot of excitement for a few years and then disappear, behavioral finance is only getting stronger. Some of the best and most compelling work has appeared in the past five or ten years.

Eventually we may even be able to drop the qualifier, behavioral, and just say finance. But honestly, we’re not there yet. There’s still going to be several years of debate and careful study before we get to that point.

Q: How do you decide which departures from rationality are most important in finance?

It’s a key question. Within behavioral finance, we all basically agree that irrational thinking is important in markets, but beyond that there are, again, vibrant, feisty disagreements about which form of irrationality is most important.

One reason we’ve been able to make a lot of progress in behavioral finance and behavioral economics is because we’ve essentially imported a lot of the ideas of two of the most important psychologists of the past half century, Daniel Kahneman and Amos Tversky. In the 1970s and the 1980s, Kahneman and Tversky documented important ways in which we systematically deviate from full rationality. Their Nobel Prize–winning framework, called prospect theory, includes concepts that are crucial to behavioral finance, including loss aversion and probability weighting.

However, I confess that, at some point, deciding which ideas from psychology might be most important in finance involves following a personal hunch about where to devote your time and effort. It’s a difficult choice to make. My hunch is that overextrapolation and ideas from prospect theory are important. But ultimately, this is science, and we just have to wait and see what the data say in the years ahead.

Q: Your work has focused on how we form beliefs about the future and how we make decisions based on those beliefs. Would you explain some of that work and where it fits in the field?

The idea that might be most important about how we form beliefs is overextrapolation. It’s just saying that when we form beliefs about the future, we put too much weight on the recent past. If the recent returns on an asset have been good, we’re too quick to think that the future returns will be good. If they’ve been bad, we’re too quick to think they will continue to be bad.

In my work, I’ve argued that this simple idea can shed light on a lot of key puzzles. One of them is excess volatility—the finding that, historically, the stock market has moved around more than can be explained by simple, rational models of investor behavior. This is Robert Shiller’s most famous discovery—what won him the Nobel Prize. Overextrapolation may also explain bubbles.

I’ve also done work trying to understand where overextrapolation comes from. I argued, in one of my first papers, that an idea of Kahneman and Tversky’s called representativeness may be an important driver of overextrapolation. When we see a good economic data point, we leap to the conclusion that it represents a positive trend. Representativeness is a possible source of this overreaction in financial markets.

This proposal that representativeness underlies overextrapolation, which in turn may be a fundamental cause of excess volatility and bubbles, remains an active topic. Some of the cutting-edge research papers today continue to argue that representativeness is an important driver of excessive fluctuations in financial markets.

Q: What about the decision-making side?

The decision-making side is strongly influenced by ideas in Kahneman and Tversky’s prospect theory framework. The most famous one is loss aversion—the idea that humans tend to be more sensitive to potential losses than to potential gains. Building on earlier work, including work by Richard Thaler, I’ve argued that loss aversion may help explain why a lot of people are reluctant to invest in the stock market, even though it would probably be a good idea to do so. And even if they do invest in the stock market, they don’t invest as much as they probably should.

The logic is quite clear. You think to yourself, “If I invest in the stock market, the stock market could go up, and that would feel good. But it could also go down. If that happens, I’m going to feel awful. So I’m just not going to invest.”

Again, it’s a straightforward, believable idea. There’s now evidence that loss aversion is a significant influence on people’s participation in the stock market.

Q: Your work on IPOs highlights another aspect of prospect theory. Would you describe that research?

Probability weighting is a less well-known aspect of prospect theory, but I think it may actually be more important in financial markets. In the traditional rational view of decision making, if you think some outcome is going to occur with probability 0.2, then you should put a weight of 0.2 on it in your decision making; if you think some outcome is going to happen with probability 0.7, then you should put a weight of 0.7 on it. But Kahneman and Tversky and others found this remarkable thing: the brain actually weights probability in a non-linear way. We overweight low-probability outcomes. You can see this tendency in the popularity of lottery tickets.

One thing I’ve shown in my work is how the strong preference for lottery-like payoffs unlocks a puzzle about IPOs. On average, over the five years after an IPO the stocks have fairly low returns. Why? Well, one explanation is, IPOs are like lotteries. Most of them don’t pay off, but a few have amazing returns. If we’re excessively drawn to IPOs because we think of them as lotteries, we end up overpaying, which leads to the subsequent average returns being low.

The nonlinear, irrational way we do probability weighting can explain a number of other important phenomena as well, which is why I say it may be even more useful than loss aversion when thinking about financial markets.

Q: What led you to academics?

For reasons that will forever be mysterious, I was excited about being an academic from a young age. There weren’t any academics in the family. I didn’t really know what it entailed, but I found it very exciting to imagine being an expert and discovering something new.

I was interested in many different fields but going into university, at Cambridge, I picked math. I enjoyed it tremendously, even when it was difficult. I remember a course in number theory. I spent many, many hours trying to make sense of all the theorems and how they were connected. When I finally understood, it was such a great moment. There’s a very profound satisfaction—you could even call it a type of happiness—that comes from figuring out something that’s true about the world.

I considered a PhD in math, but decided against it after talking it over with my parents. I grew up mainly in South East London in a Greek family that moved to the U.K. for my dad’s work in maritime shipping; they’re all back in Greece now. There’s a strong practical streak in my family that has been an important influence, so I was interested in something more applied. During the summers, I’d worked in investment banks. That gave me the idea to do a PhD in financial economics. It was a quantitative field that was also more applied. That’s important to me. I really want to be saying something about the world.

Q: Why did you choose Yale SOM?

It was a natural fit. You could say that modern behavioral finance started at Yale through the work of Robert Shiller. Indeed, when I moved to Yale, one of my goals was to continue the tradition of the work that he began and make the school a center for behavioral finance research.

With the help of colleagues, through teaching, conferences, and hiring, we’ve tried to do that. Ultimately, it’s for others to judge, but I think we’ve succeeded.

Q: In addition to research, what goes into developing a new field?

There are three other areas of work that I value highly: advising doctoral students, trying to build up institutions that can foster and encourage behavioral finance research, and expanding awareness of the field beyond academia.

Supporting the development of the next generation of researchers is part of developing a discipline. As a doctoral student, I benefited tremendously from the mentorship of my advisors. Like many other people, I struggled at times during my PhD. You can be smart and hardworking but when you’re trying to create new knowledge you can’t fully control whether inspiration will strike at the right time.

At a point where I was really concerned it might not work out for me, my phone rang. It was one of the faculty members at Harvard, the late Gary Chamberlain. He simply said, “I was just thinking about you, Nicholas. That line of research we were talking about, it could be really fruitful, and I encourage you to pursue it.” So I did.

Over time, he spent countless hours with me, but that call was so fortuitous. What I needed in that moment was someone to offer support and encouragement. It let me go on to write something that was interesting and that people were excited about. I could be an academic after all. I’m so grateful, and I recognize the luck involved. So I care a lot about advising doctoral students. I consider it a duty, but, to an extent that I hadn’t fully predicted earlier on in my career, it has become such an important and meaningful thing for me.

I also work on institution building. I co-founded the Behavioral Economics Annual Meeting with Ulrike Malmendier of Berkeley and Ted O’Donoghue of Cornell. I also started and particularly enjoy the Yale Summer School in Behavioral Finance. More than 300 graduate students have passed through the program; many of them have gone on to do wonderful work in the field.

Finally, in 2015, I took over the NBER Behavioral Finance Conference, which was started in 1991 by Robert Shiller and Richard Thaler. They ran it for an incredible 24 years. I’ve tried to continue their work. I feel all of these programs are important to the health of the field going forward.

Q: You also mentioned communicating beyond academia.

Yes, it’s another reason Yale SOM is such a good fit. Teaching MBA students is a way to speak to practitioner audiences. With such wonderful, high-quality work being done in behavioral finance, I worry sometimes that it’s too much academics talking to other academics. The insights are important and interesting, potentially, to millions of people around the world. In addition to teaching MBA students, I accept as many invitations as I can to talk with wider audiences, but I could do more. Everyone in the profession can do more.

Q: What values drive you?

Doing high-quality work. That’s just so important, and I’m so glad to be at a university that emphasizes that. If you’re not doing your work carefully, then you’re probably not creating new knowledge; you’re just adding noise. On the other hand, understanding something true about how investors think and how financial markets work, then laying it out clearly for other people so that they can benefit, leads to a profound feeling of having done something useful.

In the process of doing the highest-quality work and saying something true, I also want to have positive relationships and rich connections with people along the way. It has always been important to me to try to interact in a positive, inclusive, friendly way with everyone that’s doing this work alongside me.