Can AI Replace Human Debt Collectors?‌

When the home audio company Sonos released a disastrous update to its app last year, one of the agents fielding angry calls from customers was an multi-modal chatbot from the AI startup Sierra. Sonos is one of many companies that are using AI for customer service, taking advantage of the technology’s growing ability to successfully navigate unpredictable conversations with human beings. ‌

If AI can handle being on the receiving end of a tricky conversation, how about initiating one? A leading consumer finance company in China is supplementing its human collection agents with AI. Can artificial intelligence get people to do something they don’t want to do, like repaying a past-due debt?‌

A new study co-authored by Yale SOM’s James Choi says no—at least, not as well as humans can. Choi and his co-authors found that no matter how much the AI improved (the company introduced several upgrades to its software during the relevant time), it could not outperform humans. When humans took over collection efforts from AI agents six days past the payment due date, they were not able to close the repayment deficit opened up by their non-human counterparts. Borrowers who were initially contacted by AI had repaid less of their initial late payment a year later and were more likely to miss subsequent payments than borrowers who only received calls from human collection agents.‌

“We can attribute some of the underperformance of AI to measurable differences in its strategies relative to human strategies, but there’s a gap that we cannot explain,” says Choi, a professor of finance. “We argue that it is the AI-ness that leads to a willingness of human borrowers to break promises to repay. Even if the AI is really, really good, if I know the thing I’m talking to is not a human being, it makes a difference.”‌

The working paper, based on 22 million cases that entered debt collection between April 2021 and December 2022, was co-authored with Yale SOM PhD student Dong Huang, Zhishu Yang of Tsinghua University, and Qi Zhang of Shanghai Jiao Tong University. During the study period, which predated the wide release of generative AI technologies such as ChatGPT, the company’s debt collection efforts assigned nearly all newly delinquent debts under 300 yuan (about $42) to AI callers while larger debts were assigned to humans. ‌

Comparing debts just above versus just below the 300 yuan threshold, Choi and his colleagues find that the value of repayments collected by AI callers during the first 30 days past due is 9% less than the value collected by humans; although the gap diminishes over time, AI callers have collected 5% less even a year later. ‌

For some larger debts, the company randomly assigned half to be called by AI for the first five days before being switched to human callers, and the other half to be called by humans from the beginning. The data show that AI callers are less able than humans to extract verbal promises to repay from borrowers, and promises that are made to AI are broken more frequently. Once humans take over from AI on day 6, they can make up some of the ground lost when AI callers initiate the collection efforts, but not all of it. ‌

“Even when humans swoop in and do the work for 360 days afterwards, they cannot reverse the damage done by that initial AI contact,” Choi says.‌

The research has important implications, especially for roles that involve persuading someone to take an action that has some personal cost.‌

There are many positions where “arm twisting” is “a really important part of the job,” Choi says. “It’s going to be hard to outsource that to AI.”‌

In a future where “AI becomes ubiquitous,” Choi adds, the research “points to a possible area where humans will retain a competitive advantage. The importance of these people skills is not going to fade away.”‌