Listeners Glean Emotions Better from Voice-Only Communications

By Jyoti Madhusoodanan

Speedy internet connections and cheap video calling have made face-to-face interaction with far-flung colleagues easier than ever. But a new study suggests that even in a digital age, audio-only conversations offer clear communication benefits: listeners tend to be more accurate at gauging speakers’ emotions during a voice-only interaction.

The results, published in American Psychologist, are among the first to suggest that our tone of voice—not facial expressions—may be the primary means by which we reveal emotions when we speak. (Read the study in American Psychologist.)

Body language and facial expressions have been extensively studied for the emotions they convey in conversations. But that’s precisely why they can be more deceptive, explains study author Michael Kraus, a psychologist at the Yale School of Management. “There’s now a lot of discussion about how to look more confident, or how to hide certain less desirable emotion states by using non-verbal communication,” he says. “There is a chance that people might mislead listeners with their nonverbal communication. Misleading people through vocal expressions is more unlikely because controlling vocals is much harder to do.”

In previous studies, researchers have found that people are better at reading emotions when presented both audio and facial expressions than when they’re asked to observe facial expressions alone. But how voice-only communication ranked was unclear.

To understand that, Kraus and his colleagues recruited study participants online and presented them with short videos of a group of friends talking and teasing each other over a nickname. Participants were given one of three versions: one group watched and listened to the video, a second only heard the interaction, and a third group only saw the video but did not hear the voices. Then the audience was asked to estimate what emotions they thought the friends were experiencing, by rating feelings such as amusement, embarrassment, or happiness on a scale of 0 to 8. People who only heard the interaction—but did not watch the video—made more accurate estimates of what the friends were feeling, Kraus found.

In a subsequent experiment, the researchers recruited undergraduate students to come to the lab and chat with each other about their preferences for movies or TV shows, and what food and drinks they liked. The students also had similarly themed conversations in a darkened room. Then, they were asked to rate their own and their partners’ emotions during both exchanges. The researchers found that participants who couldn’t see each other in the darkened room fared better at reading their partners’ emotions.

Finally, the researchers presented online participants with a digital voice reciting the friends’ teasing interaction from the prior study. If people were gauging emotional content based on the kinds of words being used, they would glean the same information with the digital voice, Kraus explains. But the artificial voice was the worst. “The difference between emotional information in voice-only communication by a computer versus a human voice was the largest across all studies,” he says. “It’s really how you speak—not just what you say—that matters for conveying emotion.”

One reason the voice is so effective at conveying emotion may be that speakers are less likely to be able to alter their tone to disguise their feelings. Another possible explanation stems from our cognitive capabilities. When communicating across multiple modes, a listener must focus on many kinds of information at once: facial expressions, words, body language, and the speaker’s tone. “It’s difficult because you might be switching attention across those channels in order to perceive emotion,” Kraus says. “Whereas if you focus on any one that has the necessary information you’d be most accurate. Our research points to the voice as the most viable channel for emotion perception.”

The results underscore the importance of listening, a skill that’s going to be increasingly important as workplaces grow more global and more diverse, according to Kraus. For managers, listening effectively can help them understand when an employee is unhappy or anticipate the needs of a business partner sooner. “There’s an opportunity here to boost your listening skills to work more effectively across cultures and demographic characteristics,” Kraus says. “Understanding other people’s intentions is foundational to success in the global and diverse business environment that characterizes both the present and the future.”