Can Online Reviews Be Trusted?
The online, user-generated review is a boon for consumers—a chance to sidestep promotional claims and get an honest assessment. But as soon as reviews appeared online, fake reviews followed. A study co-authored by Professor Judith Chevalier looks at the prevalence of fake hotel reviews and tests a hypothesis about who might post them and why.
Online shoppers have come to rely on user-generated reviews for comparing the relative merits of gadgets or getting a bit of reassurance on the cleanliness of a hotel before clicking “Buy.” But given its power to make or break a business, the online review invariably begat the fake review; the first fake reviews appeared on Yelp two weeks after the site opened. And fake reviews, says Judith Chevalier, William S. Beinecke Professor of Finance and Economics at Yale SOM, diminish the usefulness of all reviews. “It’s a problem if people suspect that the reviews are fake,” she says. “If I don’t trust the reviews, I may be hesitant to buy.”
Websites have long had algorithms for identifying and banishing individual fake reviews. In a paper in the American Economic Review, Chevalier and co-authors from the Marshall School of Business and the Tuck School of Business take a wider view, looking at the prevalence of fake reviews and testing a hypothesis about who might post them and why.
Chevalier’s study takes advantage of a difference between two major hotel reviews sites. TripAdvisor.com allows anyone to post a review of a hotel; Expedia.com only allows customers who have used the site to book a room at a hotel to review that hotel. Chevalier and her co-authors hypothesized that the owners of single, independent hotels, as opposed to chains, have the most to gain by posting a fake review—whether a positive review of their own businesses or a negative review of a competitor. They then compared the prevalence of very positive and negative reviews—specifically, five-star and one- and –two star reviews—on Expedia, where owners don’t have the opportunity to post fake reviews, to TripAdvisor, where they do. The authors found that, indeed, on TripAdvisor, independent hotels are more likely to have more positive reviews, and neighbors of independent hotels have more negative reviews.
Does that mean that reviews can’t be trusted? The results suggest that even for these hotels, most reviews are likely to be genuine—on average, the hotels in the study have 120 reviews, of which six or seven appear to be fake. “We think that certainly south of 10% of the reviews are fake,” Chevalier says. “The majority of the reviews are accurate—or at least real. Maybe they’re not accurate; they don’t necessarily accord with what your experience would be, but they’re real.”
Is posting fake reviews likely to be an effective strategy for a hotel owner? Well, Chevalier says, “it’s not honest, and it’s illegal.” But, yes, it probably works. This study did not examine sales, but other research suggests that reviews have a direct effect on sales. “My best guess, based on what I know about the industry and the mechanism, is that it’s probably likely to be effective. A one-star review of a hotel is probably a powerful deterrent for people to stay there.”
“Promotional Reviews: An Empirical Investigation of Online Review Manipulation” by Dina Mayzlin (Marshall School of Business, University of Southern California), Yaniv Dover (Tuck School of Business at Dartmouth), and Judy Chevalier (Yale School of Management) appeared in the American Economic Review in August 2014.
Related: What Makes a Good Online Review?