A Passionate Startup Pitch Is Powerful—But Can Be Misleading

Venture capitalists are looking for firms that can grow, turn a profit, and eventually be sold or go public. They have to evaluate many factors, including the validity of a company’s business plan, the market fit, and the quality of the startup team. But a new study coauthored by Song Ma, an assistant professor of finance at Yale SOM, and Allen Hu, a doctoral candidate at the school, shows that investors can be unduly swayed when people pitching startup ideas appear enthusiastic, excited, and warm—even though these qualities revealed in the pitch do not lead to later success. The work may help unpack some of the ways that human biases affect economic decision making.

To study the issue, Ma and Hu collected more than 1,100 pitch videos that entrepreneurs submitted as part of their applications to a business accelerator. The researchers split each video into three streams of information: “visual,” which consisted of a series of still images captured at a rate of 10 frames per second; “vocal,” which consisted of the presentation’s audio; and “verbal,” which consisted of the presentation’s transcript. Using a novel machine-learning approach, they then assessed the level of enthusiasm present in each of these data streams.

In the past, such an analysis “would have been nearly impossible,” Ma says. It would have required hiring individuals to review each of these streams and rate them as consistently as possible across the variables of interest. How positive did Team A look? How energetic did Team B sound? How warm was the diction of Team C?

The researchers instead asked a computer to make those judgments. “New machine learning technology has really helped us,” Ma says. “Using several different algorithms, we essentially created a person who doesn’t get tired, who can keep a consistent standard, and who rated things accurately.” Every video was analyzed on the visual, vocal, and verbal dimensions, and these ratings were then compiled into a single Pitch Factor representing the video’s overall positivity.

The researchers found that teams that looked happier and more positive were 17% more likely to receive funding; teams that sounded more enthusiastic were 27% more likely to receive funding; and teams that used warmer and more engaging language were also more likely to be funded. (On the other hand, teams that discussed their own ability or competitiveness were penalized.) Overall, an above-average Pitch Factor increased the likelihood of investment by 35%.

Once they hit the market, though, these same firms underperformed, on average, based on key indicators like whether they raised a second round of funding or stayed in business. In essence, an enthusiastic presentation overshadowed a lackluster business plan.

This effect was especially pronounced for teams composed entirely of women. In those cases, investment decisions were seven times more sensitive to the performance of the pitch. The researchers speculate that investors simply reward women who fit their stereotypes—women should be warm and positive—and avoid those who do not. For teams comprising both genders, the performance of the male appeared to drive investment decisions, suggesting that women are simply ignored when they present alongside men.

Ma sees two possible explanations for the overarching result. “It’s possible that investors incorrectly believe that more enthusiastic people actually have higher-quality ideas, which would simultaneously explain the higher investment rate and lower performance,” he says. “Or, this could be an example of something that’s well-documented in psychology where we simply like to be around people who are positive.”

The researchers ran a lab experiment to distinguish between these two possibilities. They showed a handful of accelerator pitch videos to MBA students and asked them where they would invest, along with their levels of confidence and expectations of startup success. From these survey questions, the researchers were able to elicit information on why the participants made the decisions that they did. In the end, people overwhelmingly—and incorrectly—associated positive presentations with stronger performance.

For entrepreneurs, the implications of these findings are obvious: remain as upbeat as possible in pitch meetings. For investors—and others who evaluate people based on how they present themselves and their work—the takeaway is murkier. How do we control for biases that slip below recognition?

These biases are especially strong in situations where people have limited time to complete a large-scale screening, Ma notes. “If you have 1,000 applicants and you need to quickly drop 800 of them, this is where our biases can be particularly problematic,” he says. “The first thing to do is recognize when you’re in a situation like that—reviewing résumés to fill a job opening, for example. In those case, think about whether you can use more consistent and robust machine learning-driven algorithms to support the work.”

More speculatively, Ma suggests that pulling together a diverse group of people to serve as raters and reviewers could introduce a diverse set of biases that essentially cancel each other out. Finally, if one of the three information channels exerts the most influence — something that Ma is investigating—then safeguards could be put in place to protect against that avenue of bias. Just as orchestras conduct blind auditions in order to counteract bias based on gender or race, perhaps investors could devise a system of pitching that reduces interference from subjective variables and shines a stronger spotlight on the objective potential of the business plan.

“If we found that voice is useful but visuals are distracting, then perhaps we should encourage people to speak over the phone in the first round of interviews, or investments,” Ma says. “This interaction bias is very important, and we should look for ways to reduce or counteract it.”