The power of competition to spark greater effort and focus is well known. Ovid wrote about it in his tract on love, completed around 2 AD: “A horse never runs so fast as when he has other horses to catch up and outpace.” Companies have long used competition to spur innovation, pitting internal teams against each other in search of the best idea; political parties wield it in the form of primaries to find candidates with the broadest appeal.
Using competition this way gives organizations flexibility when the best approach to some future challenge is unclear. But it also has its drawbacks, among them the problem of wasted effort: two or more teams working on different projects toward the same end are not as efficient as a single, larger team working on one project toward that same end.
“This is a classic economic tradeoff between competition and collaboration,” says Joyee Deb, associate professor of economics at Yale SOM. Competition provides diversity of roughly developed ideas, allowing adaptation to changing circumstances. But the benefit of flexibility must be balanced with the efficiency loss of wasting productive effort on the wrong approach. Collaboration allows efficient development of a single well-refined idea. “The question we wanted to ask was how an organization might find the optimal balance between competition and collaboration.”
Deb and two colleagues, Aditya Kuvalekar of the University of Essex and Elliot Lipnowski of Columbia University, model a firm trying to decide between two projects. In the model, a firm must evaluate the two projects as they are developed, and needs to pick one of them before a deadline. The firm wants to pick the more profitable project, while the teams have conflicting interests, in that each wants its own project to be chosen.
“We found that the optimal thing to do, in fact, looks a lot like what we see in real life,” Deb says. “Start with an initial phase of pure competition between teams, to be always followed by a period of collaboration.”
The researchers propose three principles that should guide this process. First, competing ideas should be compared by relative performance. If a company has two teams working against each other to develop a new product or service, decision makers shouldn’t spend time evaluating the general market prospects of each. What matters is the promise of one team’s product or service relative to the other team’s. One team’s product can be picked once it is sufficiently ahead of the other.
Second, the decision to switch from competition to collaboration rests on what Deb calls a “decreasing lead threshold.” For a company to invest in Project A over Project B very early in the competition requires Project A to be significantly more promising, relatively speaking. As the competition wears on, the required size of this lead diminishes; close to the project deadline, it becomes very small. “Intuitively, you need a big lead at the beginning because you’re giving up the option of another project or person,” Deb says. “You then reduce this threshold over time.”
“Do all this learning quickly in the beginning and then invest in one person or idea. Don’t go back and forth between competition and collaboration.”
Finally, the switch from competition to collaboration must be irreversible. Competition comes first, and should be used to learn about the relative merits of a person or project and to inform a decision about which to pursue. Once the decision is made, however, there is no looking back. “Do all this learning quickly in the beginning and then invest in one person or idea,” Deb says. “Don’t go back and forth between competition and collaboration.”
This absolute commitment is important not only because it maximizes the time devoted to collaborating on the leading project, but also because it is necessary to get collaboration off the ground. If Joe Biden were nominated but the option remained open to replace him with another contender—Sanders, Warren—the other contenders would not be willing to lend their support to Biden’s campaign in the hope that they may get another chance. Likewise, if a company selects one team’s approach to a problem as the right one but leaves open the possibility of reconsideration, the other teams will not invest fully in collaboration.
Deb and her coauthors also emphasize, however, that while competing first and then collaborating based on the relative merits of a project will generally help firms choose the best project, it’s still possible to make mistakes. Given this caveat, she is frequently questioned about the wisdom of fully foreclosing other options. Is it really best to move ahead without looking back?
“The paper shows that the best you can possibly do is to commit to such a two-phase rule: a temporary phase of competition followed by collaborating on the winner without looking back again,” she says. “Fostering collaboration increases the value of the firm’s chosen project because all teams end up collaborating to work on it. Big mistakes can happen sometimes, but the chance of such mistakes is lower for projects with longer collaboration.”