Cornell researchers and Crisis Text Line have developed a way to study how volunteer counselors improve at difficult mental health crisis conversations, with the goal of identifying people who may benefit from additional training and support.
The method uses natural language processing to examine how counselors respond when a conversation begins to deteriorate, according to Cornell’s account of the research. It then looks for changes when the same counselor encounters a similar situation later.
The work is a feasibility demonstration that requires more testing before it can guide counselor training. The researchers say it is intended to support human counselors, rather than replace them with AI.
Researchers compared early and later conversations
Vivian Nguyen, a Cornell computer science doctoral student, led the study with senior author Cristian Danescu-Niculescu-Mizil, a professor of information science in Cornell’s Bowers College of Computing and Information Science.
Other authors were Cornell computer science professor Lillian Lee and Elizabeth Olson, Crisis Text Line’s vice president for AI and natural language processing clinical research science.
Nguyen presented the work, “Adapting from Downturns: Prediction of Long-Term Conversational-Skill Development in Mental-Health Crisis Counselors,” at the Conference on Empirical Methods in Natural Language Processing, held Oct. 24-29 in Budapest, Hungary.
The team analyzed more than 1.5 million deidentified Crisis Text Line conversations from January 2015 through October 2020. Personally identifying information was removed by the platform, and models ran internally because of the sensitivity of the conversations.
Crisis Text Line is a nonprofit providing free, confidential mental health support. Its volunteer counselors receive standardized training on handling conversations, but that training does not give explicit guidance on how their skills should develop over time. Two Cornell co-authors completed the training to understand the work firsthand.
For counselors with at least 125 conversations, the study compared the first 50 conversations with the most recent 50. The selected counselors had spent an average of about 20 months on the platform.
An internally hosted language model rated how conversations ended on a scale of 1 to 5. Higher scores reflected more positive endings, including a texter becoming calmer or feeling supported; lower scores included abruptly leaving a conversation or remaining distressed.
Difficult moments offered a stronger signal
The system identified conversational “downturns,” when its forecast of how an exchange would end worsened sharply. A counselor or texter might not recognize the moment themselves.
Researchers then matched those moments with similar situations later in the same counselor’s work. Another model summarized changes in the counselor’s responses, allowing the team to study patterns of adaptation.
The team also examined alternatives, including randomly chosen moments and changes in sentiment. Responses to downturns provided more information about improvement than those other selections.
The research paper describes a balanced prediction sample of 1,488 counselors and 148,800 conversations. It compared counselors in the top and bottom quarters of improvement who had similar starting performance.
Across the longer-serving counselors studied, 36% did not improve on the conversation-outcome measure.
The full approach predicted future improvement with 61.9% accuracy, compared with 57.5% for its strongest raw-message baseline. That modest gain differs from the separate task of detecting improvement after it has already occurred.
More testing is needed before training use
The researchers propose using the approach eventually to help platforms direct training and supervision toward volunteers who remain active but are not improving through experience alone.
Their outcome measure is a narrow indicator of conversational quality, rather than a measure of lasting mental health recovery. Abrupt disengagement can sometimes reflect unrelated events, such as a phone losing power.
The findings apply to the platform’s longer-serving counselors. Testing across counselor subgroups, different cohorts and time periods remains necessary before operational decisions.
The underlying counseling data cannot be made public because of its private nature. The team released code and a demonstration using public data to support further research.
The work was supported by the National Science Foundation, the Cornell Center for Social Sciences and a 2026 GEM Fellowship for Nguyen. The paper also identifies support to Crisis Text Line from the Jensen and Lori Huang Foundation.
FingerLakes1.com is the region’s leading all-digital news publication. The company was founded in 1998 and has been keeping residents informed for more than two decades. Have a lead? Send it to news@fingerlakes1.com.
