Researchers from the University of Johannesburg (UJ) have published a promising approach to early detection of people at risk of suicide-related behaviour (SRB) in the journal Scientific Reports.
The study describes using an AI algorithm, routine patient intake data, and light-weight computing to estimate SRB risk following admission to a hospital emergency room (ER). The AI model was built with historical open-source patient data from the USA. It is intended to assist ER staff rather than replace them.
Resource constraints in South Africa are a major reason to develop an AI model such as this, says lead researcher Mr Tsholofelo Mokheleli.
Late at night and over weekends, some South African hospital emergency rooms can see one ER staff member with 30 or more patients to triage, he adds.
Mokheleli is an assistant lecturer at the Department of Applied Information Systems within the UJ College of Business and Economics (CBE). The study forms part of his PhD research.
During busy ER shifts, an objective, consistent measure to help staff rapidly screen patients could be helpful for early detection of SRB risk.
"The AI model will flag that a patient is at risk and provide reasons. A health care professional still needs to take the final decision however, the clinician must do a severity scaling to determine if the person is really suicidal or not," says Mokheleli.
Existing SRB screening is resource intensive
The current two screening methods for SRB used in South African ERs require much time and effort from clinicians, says Prof Nompumelelo Ndlovu.
Prof Ndlovu is an Associate Professor in the Department of Nursing, within the Faculty of Health Sciences at the University of Johannesburg. She is a co-author of the study and one of the co-supervisors of Mokheleli's PhD research.
When anyone first arrives at an ER, the first priority is to stabilize them medically, she adds.
Even for conscious patients admitted to the ER, the standard mini-mental status examination for SRB risk can take 15 minutes and varies between patients, she says. A full mental status examination and history taking for one patient at high risk of SRB could consume a day or more from a mental health professional at the hospital.
"Not everyone who goes into a medical institution has mental health issues. What we are trying to do, is to find those people who do not even say it out loud," says Mokheleli.
Adds Prof Ndlovu: "Some people do not realize they are at risk of SRB. People may not be aware of what is happening with them unless they speak to somebody. But it is not an easy topic to speak about. There is so much stigma. So how safe do you feel to tell the next person about what is happening with you?"
Routine admissions data, no new ER workload
The study's AI model identified patient age, pain, and a prior mood disorder recorded at the same hospital as the major reasons for flagging a patient for SRB risk. Vital signs such as temperature and blood pressure follow on that, with prior substance-use and anxiety diagnoses playing smaller roles.
"The USA patient admission information we used to build this model is similar to what you would get in South Africa," says Mokheleli.
"It includes demographics, age, gender, vital signs, temperature, heart rate, respiratory rate, oxygen saturations, and systolic and diastolic blood pressure. From that information, we calculate physiological features like the patient's shock index and pulse pressure," he adds.
"Another aspect is the patient's prior history at the same hospital. The model checks if the patient has been at the same hospital before and what the problem was. If there was some kind of substance abuse, anxiety or depression documented with ICD-9 or ICD-10 codes on a previous visit, then the model takes that into account. Night and weekend admissions are also included, as well as notes from medical staff about self-harm on previous visits."
Local records, local patient data governance
Because the AI model is built on existing electronic patient admissions data, ER staff would not need to do additional data capture for future system implementations. However, in hospitals with paper-based records, the requirement for digitised patient records in a database would be a barrier.
A major reason for building the AI model on routine patient admissions data, is data governance, says co-author and Mokheleli's PhD main supervisor, Prof Patrick Ndayizigamiye. Prof Ndayizigamiye is a visiting Associate Professor in the Centre of Applied Data Science, at the UJ College of Business and Economics.
"The question is always 'who owns the confidential patient data and where is it stored?' Routine admission data belongs to the patient, and it is stored within the hospital's systems already," he adds. "As opposed to asking a cloud storage vendor to put the data in their server, which generates a lot of data governance issues."
False negatives and false positives
With this AI model, a number of factors influence how many patients it will flag correctly for SRB risk. The factors include the size of the dataset; the dynamics of the patient population at the hospital from which data was selected; how few patient records in the training dataset recorded SRB risk; the value of the SRB risk decision threshold; and the AI algorithm selected for analysis.
The same factors also influence how many patients truly at risk for SRB the model misses.
Per 1000 emergency department visits, using the 41,202-patient dataset of the study, the model correctly predicted 866 patients as not at risk of a documented SRB outcome in the 30 days after admission to ER (true negatives).
Per 1000 visits, the model incorrectly predicted or missed 13 cases as not at risk of SRB, although an SRB-related outcome was documented within the 30 days after admission to ER (false negatives).
Per 1000 visits, the model flagged 121 patients as potentially at risk of SRB, and requiring further screening by clinicians (true positives plus false positives), a resource-intensive task.
Of the 121 visits per 1000 the model predicted SRB risk for, 23 had a documented SRB-related outcome that was recorded within the 30 days following the ER admission (true positives).
Of the 121 visits per 1000 flagged for SRB risk, 98 flagged cases did not have any SRB-related outcome documented with ICD-9 or ICD-10 codes in the same hospital's records, in the 30 days after admission to the ER (false positives).
Source of patient data
Finding a suitable digital patient admissions dataset in South Africa proved difficult, says Prof Ndlovu. To solve this, the study is based on an open dataset from a single hospital in the USA, extracted from the MIMIC-IV resource. The hospital had a mainly White and Hispanic patient population, which is completely different from South Africa.
The Massachusetts Institute of Technology's (MIT) Laboratory for Computational Physiology in collaboration with Beth Israel Deaconess Medical Centre developed MIMIC-IV. It contains anonymous records from patients admitted to hospitals and ICUs within 24 hours of visiting an Emergency Room (ER) in the USA between 2008 and 2019.
The AI model in the study used MIMIC-IV's detailed information about patient age and gender, admission characteristics, vital signs and previous psychiatric diagnoses. These are routinely recorded in hospital systems and helped the model identify patterns associated with SRB.
Tuning the model, outputs for clinicians
The study's model did not include job loss, bereavement, drug, or alcohol-related observations from medical staff, but can be included in future versions, says Mokheleli.
Future models will use bigger data sets, potentially different AI algorithms, and adjusted techniques to improve the performance of the model, adds Dr Tebogo Makaba, a study co-author and Mokheleli's PhD co-supervisor. Dr Makaba is Senior Lecturer in the Department of Applied Information Systems at the UJ CBE.
Future versions of the model will also require the 'patient risk threshold' to be adjusted for each hospital, says Mokheleli, since patient populations vary.
In addition, the AI model's current output - confusion matrices and SHAP (SHapley Additive exPlanations) plots - is hard to understand.
Says Mokheleli: "A SHAP plot shows how a model made a decision about one patient. We are busy designing a user interface that clinicians can interact with and get answers they can understand."
