Being a good listener can sometimes involve more than just listening. It can also mean knowing when and how to say just the right thing - a conversational skill that's invaluable to a crisis counselor.
It's also a skill that can be learned, and researchers from the Cornell Ann S. Bowers College of Computing and Information Science and Crisis Text Line, a global nonprofit that provides free, confidential mental health support, have developed a method for tracking how counselors hone their ability to steer conversations with clients toward positive outcomes, and identifying early in a counselor's career if they are likely to improve.
"Crisis counselors regularly navigate high-stakes conversations," said lead author Vivian Nguyen '24, M.S. '25, a doctoral student in the field of computer science. "We wanted to understand how they get better at handling these conversations over time, and whether we could tell early on which counselors might benefit more from additional support."
Nguyen is lead author of "Adapting from Downturns: Prediction of Long-Term Conversational-Skill Development in Mental-Health Crisis Counselors," which she presented at the Conference on Empirical Methods in Natural Language Processing, held Oct. 24-29 in Budapest, Hungary.
The research team, led by Cristian Danescu-Niculescu-Mizil, professor of information science (Cornell Bowers), used natural language processing (NLP) to identify pivotal "downturn" moments in an online crisis conversation, where a counselor may struggle to keep a person calm and talking or texting. The method also captures how the same counselor adapted their response when faced with a similar moment in conversations later in their career.
Danescu-Niculescu-Mizil is senior author; other contributors are Lillian Lee '93, the Charles Roy Davis Professor of Computer Science (Cornell Bowers), and Elizabeth Olson, vice president for AI/NLP clinical research science at Crisis Text Line. The team charted the progress of Crisis Text Line's volunteer crisis counselors for the research.
The counselors receive standardized training, including how to handle conversations, but aren't offered guidance on how to improve over time. Two of the Cornell co-authors completed the training program as part of the research.
Nguyen and the team analyzed more than 1.5 million deidentified conversations, from Jan. 2015 to Oct. 2020, and used NLP to identify conversation outcomes based on whether a texter either abruptly ended a conversation or was guided to a more positive state. Using an internally hosted large language model (LLM), conversations were given a score from 1-5, with higher scores corresponding to more positive outcomes.
To measure improvement, the researchers compared the counselors' performance at two distinct points in their careers. Analyzing counselors with at least 125 conversations (around 20 months), they compared the first 50 from their early career with the most recent 50 from their late career.
The researchers first used a method to identify "downturn" moments - points that signaled a difficulty in the conversation, even if neither party was aware of it. The method detects at which points in the conversation the expectation about how the conversation will end takes a turn for the worse. The team also analyzed other moments in the conversations, including random ones, to confirm that responses to downturn moments yielded the most information about whether a counselor will improve from their early to late career.
Another system learned how to summarize how counselors changed the way they respond to downturn moments. The system was internally hosted due to data privacy concerns. This "learning from mistakes" method demonstrates the feasibility of their approach, which will require more testing before it can be deployed for counselor training, the researchers said.
Danescu-Niculescu-Mizil stressed that "our work is in no way aiming to replace human counselors with AI." But it is an example, he said, of how an NLP model can be used to hone counselor conversational skills.
Olson agreed. "This research opens doors to thinking about how to better support human crisis responders as they learn from experience," she said.
This research was supported by grants from the National Science Foundation, the Cornell Center for Social Sciences and a 2026 GEM Fellowship to Nguyen.
