When Michal Herzenstein and her fellow researchers discovered that word choice in loan applications can strongly predict whether borrowers will pay back the money, they ran into some initial skepticism in the financial world.
“The financial crowd is really rational,” Herzenstein said. “They would say, ‘Why would something that I say, (with) really no way to verify whether what I say is true or not … impact my behavior in the next three years.’”
However, the paper by the associate professor of marketing at the University of Delaware’s Alfred Lerner College of Business and Economics quickly garnered interest after its 2019 publication, and recently won the Weitz-Winer-O’Dell Award from the Journal of Marketing Research. This award recognizes research published in the journal five years previously that has made the most impact in the years since.
“When Words Sweat: Identifying Signals for Loan Default in the Text of Loan Applications,” by Herzenstein and two colleagues, Oded Netzer of Columbia University and Alain Lamair of the University of Texas at Austin, was one of two papers to win the award this year.
“The paper has garnered over 230 citations and nearly 6,000 downloads and was even featured in the bestselling book, ‘Everybody Lies’ by Seth Stephens-Davidowitz. More importantly, the paper has broad appeal, with 63% of its citations coming from outside the field of marketing,” the journal’s award committee said when it announced the recognition in April.
“We were excited to be nominated for this,” Herzenstein said, noting it is a top award from the top journal in the marketing field.
It’s certainly garnered more interest than any other research she’s done so far, she said, estimating they have spoken about their work at many invited talks and conferences around the world.
“This level of dual impact—on both research and practice—is rare,” the award committee noted.
The authors have also shared insights with the financial industry, although Herzenstein didn’t have details on what exactly they’ve done with the findings. The industry is notably close-mouthed about how it uses this kind of information in its predictive algorithms.
One way the findings are useful is that they predict future behavior, whereas a credit score and other financial measures (like debt to income ratio) contain older information about the past, Herzenstein said. For example, an applicant might mention medical bills when those have not yet shown up on the credit score.
Would-be borrowers who talk about the short term in their applications are more likely to default, in another example. And other word usage like that seems innocuous, like using “rather” or “I,” can also be predictive.
Herzenstein is still intrigued by the phenomenon. She and Netzer have collaborated with other researchers on a paper called “The Language of (Non)replicable Social Science,” which is forthcoming in Psychological Science, a top journal in psychology.
Replication, or the ability to repeat an experiment in a different lab with the same results, is a key component of scientific study, and Herzenstein and her colleagues found that papers that can’t be replicated tend to be “written in a way that seems to allude to the possibility the researchers were cutting corners.”
There are a lot of reasons why a paper would not be replicable, Herzenstein said, but these papers are generally more abstract and have fewer details, perhaps indicating that the writers are trying to convince the reader of their findings, rather than letting the results speak for themselves.
Seeing the way simple word usage can predict so much, Herzenstein says, “just blows my mind.”