AI Photo Analysis Illuminates How Personality Traits Predict Career Trajectories
Recent advancements in AI have made it possible to infer personality traits from a single photograph of a person’s face. A new study co-authored by Yale SOM’s Kelly Shue applies these techniques to a large set of photos of MBA graduates to assess the effects of personality on labor market outcomes.
Personality characteristics have long been thought to predict a wide range of personal outcomes, including career success: being conscientious, for example, will take you further professionally than, say, being neurotic.
However, because personality has primarily been measured through surveys, obtaining large-scale, detailed data on personality traits has been challenging. Furthermore, these surveys are often given in high-stakes situations, such as when applying for a job, and people may be motivated to answer the survey questions so as to showcase traits valued by an employer or school. “That’s a major problem with the surveys,” Yale SOM’s Kelly Shue observes. In recruitment settings, for example, “it’s pretty obvious that you want to seem more conscientious.”
So Shue was intrigued by a 2020 paper that used AI to analyze facial features taken from over 12,000 volunteers who submitted their photos and also completed a personality survey. It found that facial features were reliably correlated with the “Big Five” personality traits: openness, conscientiousness, extraversion, agreeableness, and neuroticism. While the exact mechanism underlying the connection between personality characteristics and facial features is hard to pin down, there is scientific research showing that these traits can be connected through genetics, hormone exposure, and how people perceive and respond to facial traits. Shue and her co-authors realized they could harness AI’s powerful capabilities over a large dataset to learn more about the relationship between personality types and labor market outcomes.
Inferring information from facial features is an area in which AI appears to perform particularly well. Studies show that when multiple people assess a stranger’s personality based on a photo, their responses tend to vary widely and have low correlation. “I do think the machine is picking up on many things that are too subtle to be consistently picked up by the human eye,” Shue says.
But such a form of assessment may trade accuracy for ethical concerns. “This kind of new technology is so exciting to us from a research perspective, because it gives us way more data and power to explore the associations between personality and career outcomes,” Shue says. “But we are very much not advocating that this technology be used by firms as part of their hiring process.”
For their new research, Shue and her co-authors, Marius Guenzel and Shimon Kogan of Wharton, and Marina Niessner of Indiana University, leveraged AI’s predictive power to analyze the personalities of 96,000 graduates of top MBA programs from their LinkedIn photos. They combined this with additional data from LinkedIn, going back about 20 years, on estimated earnings, turnover, and promotions. They show that AI’s personality assessments extracted from LinkedIn photos (what they call the “Photo Big 5”) have substantial predictive power for outcomes such as the prestige of an individual’s MBA program or earnings upon graduation.
“We can estimate these people’s personalities, and then we can test some of the theories from existing research that personality predicts labor market outcomes, but using a much larger and more detailed sample,” Shue explains.
Specifically, the researchers found that moving from the bottom to the top quintile of desirable personality traits (the exact mix of which differs slightly for women versus men) comes with improvement in a person’s predicted school ranking of 7.3% for men and 17.3% for women.
Going from the bottom quintile to the top quintile of desirable Photo Big 5 personality traits is also associated with an increase in the predicted compensation for a first post-MBA job of 8.4% for men—a larger effect than the pay gaps between Black and White men (3.5% gap) or Asian and White men (a 1.9% gap) among MBA graduates. These personality traits appear to be distinct from attractiveness and cognitive ability measures such as school rank, GPA, or test scores. Even after accounting for these factors, the Photo Big 5 remains a significant predictor of labor market outcomes.
As detected by the AI facial analysis, specific traits have divergent predictive effects by gender: agreeableness positively predicts school rankings for men, but negatively for women. Conscientiousness predicts higher initial compensation for both genders, but is associated with faster growth in compensation for men and slower growth for women.
People are being screened on features that they can’t control and are not easily changeable.
Shue says it is not clear whether these divergent results reflect a limitation in AI facial analysis or differences in how these traits actually operate in work settings for women and men. The algorithm was trained separately for women and men, and may be picking up on traits with varying degrees of precision, Shue notes. (Because the sample was roughly three-fourths men, the statistical power is also stronger for male MBAs.) But she points to earlier research that has documented gendered behaviors and reactions in the workplace. Being more conscientious may mean, for women, being less likely to move to a different firm or to push for a promotion or raise. Or even if they do, it might backfire. Studies have shown that women who negotiate aggressively for a raise are perceived less favorably than men who advocate for themselves with the same vigor.
Shue is excited by the many research questions she and her co-authors can explore with this large data set, but, she says, that doesn’t mean an AI-based personality assessment could or should be used in the real world. “Essentially, people are being screened on features that they can’t control and are not easily changeable,” Shue explains. “If someone takes steps to improve her personality, her efforts would not be rewarded, because her change in personality may not be evident from her face.”
Of course, people have been known to modify their faces, either using digital tools or in real life. “Suppose this type of technology gets used in labor market screening, or maybe dating markets,” Shue muses. “Going forward, you could imagine a reaction in which people then start modifying their pictures to look a certain way. Or they could modify their actual faces through cosmetic procedures.”
The next step for Shue and her colleagues is to explore whether certain personality types are drawn to specific industries or whether those personality types are more likely to succeed within given industries. Even within a given industry, they’ve found, “Personality still strongly predicts pay and promotions and raises.”