Predicting adolescent suicidal behavior following inpatient discharge using structured and unstructured data
Copyright © 2024 Elsevier B.V. All rights reserved..
BACKGROUND: The objective was to develop and assess performance of an algorithm predicting suicide-related ICD codes within three months of psychiatric discharge.
METHODS: This prognostic study used a retrospective cohort of EHR data from 2789 youth (12 to 20 years old) hospitalized in a safety net institution in the Northeastern United States. The dataset combined structured data with unstructured data obtained through natural language processing of clinical notes. Machine learning approaches compared gradient boosting to random forest analyses.
RESULTS: Area under the ROC and precision-recall curve were 0.88 and 0.17, respectively, for the final Gradient Boosting model. The cutoff point of the model-generated predicted probabilities of suicide that optimally classified the individual as high risk or not was 0.009. When applying the chosen cutoff (0.009) to the hold-out testing set, the model correctly identified 8 positive cases out of 10, and 418 negative cases out 548. The corresponding performance metrics showed 80 % sensitivity, 76 % specificity, 6 % PPV, 99 % NPV, F-1 score of 0.11, and an accuracy of 76 %.
LIMITATIONS: The data in this study comes from a single health system, possibly introducing bias in the model's algorithm. Thus, the model may have underestimated the incidence of suicidal behavior in the study population. Further research should include multiple system EHRs.
CONCLUSIONS: These performance metrics suggest a benefit to including both unstructured and structured data in design of predictive algorithms for suicidal behavior, which can be integrated into psychiatric services to help assess risk.
| Medienart: |
E-Artikel |
|---|
| Erscheinungsjahr: |
2024 |
|---|---|
| Erschienen: |
2024 |
| Enthalten in: |
Zur Gesamtaufnahme - volume:350 |
|---|---|
| Enthalten in: |
Journal of affective disorders - 350(2024) vom: 01. März, Seite 382-387 |
| Sprache: |
Englisch |
|---|
| Beteiligte Personen: |
Carson, Nicholas J [VerfasserIn] |
|---|
| Links: |
|---|
| Themen: |
Adolescence |
|---|
| Anmerkungen: |
Date Completed 21.02.2024 Date Revised 09.03.2024 published: Print-Electronic Citation Status MEDLINE |
|---|
| doi: |
10.1016/j.jad.2023.12.059 |
|---|
| funding: |
|
|---|---|
| Förderinstitution / Projekttitel: |
|
| PPN (Katalog-ID): |
NLM366486942 |
|---|
| LEADER | 01000caa a22002652 4500 | ||
|---|---|---|---|
| 001 | NLM366486942 | ||
| 003 | DE-627 | ||
| 005 | 20240309232107.0 | ||
| 007 | cr uuu---uuuuu | ||
| 008 | 240108s2024 xx |||||o 00| ||eng c | ||
| 024 | 7 | |a 10.1016/j.jad.2023.12.059 |2 doi | |
| 028 | 5 | 2 | |a pubmed24n1321.xml |
| 035 | |a (DE-627)NLM366486942 | ||
| 035 | |a (NLM)38158050 | ||
| 035 | |a (PII)S0165-0327(23)01526-4 | ||
| 040 | |a DE-627 |b ger |c DE-627 |e rakwb | ||
| 041 | |a eng | ||
| 100 | 1 | |a Carson, Nicholas J |e verfasserin |4 aut | |
| 245 | 1 | 0 | |a Predicting adolescent suicidal behavior following inpatient discharge using structured and unstructured data |
| 264 | 1 | |c 2024 | |
| 336 | |a Text |b txt |2 rdacontent | ||
| 337 | |a ƒaComputermedien |b c |2 rdamedia | ||
| 338 | |a ƒa Online-Ressource |b cr |2 rdacarrier | ||
| 500 | |a Date Completed 21.02.2024 | ||
| 500 | |a Date Revised 09.03.2024 | ||
| 500 | |a published: Print-Electronic | ||
| 500 | |a Citation Status MEDLINE | ||
| 520 | |a Copyright © 2024 Elsevier B.V. All rights reserved. | ||
| 520 | |a BACKGROUND: The objective was to develop and assess performance of an algorithm predicting suicide-related ICD codes within three months of psychiatric discharge | ||
| 520 | |a METHODS: This prognostic study used a retrospective cohort of EHR data from 2789 youth (12 to 20 years old) hospitalized in a safety net institution in the Northeastern United States. The dataset combined structured data with unstructured data obtained through natural language processing of clinical notes. Machine learning approaches compared gradient boosting to random forest analyses | ||
| 520 | |a RESULTS: Area under the ROC and precision-recall curve were 0.88 and 0.17, respectively, for the final Gradient Boosting model. The cutoff point of the model-generated predicted probabilities of suicide that optimally classified the individual as high risk or not was 0.009. When applying the chosen cutoff (0.009) to the hold-out testing set, the model correctly identified 8 positive cases out of 10, and 418 negative cases out 548. The corresponding performance metrics showed 80 % sensitivity, 76 % specificity, 6 % PPV, 99 % NPV, F-1 score of 0.11, and an accuracy of 76 % | ||
| 520 | |a LIMITATIONS: The data in this study comes from a single health system, possibly introducing bias in the model's algorithm. Thus, the model may have underestimated the incidence of suicidal behavior in the study population. Further research should include multiple system EHRs | ||
| 520 | |a CONCLUSIONS: These performance metrics suggest a benefit to including both unstructured and structured data in design of predictive algorithms for suicidal behavior, which can be integrated into psychiatric services to help assess risk | ||
| 650 | 4 | |a Journal Article | |
| 650 | 4 | |a Adolescence | |
| 650 | 4 | |a Electronic health records | |
| 650 | 4 | |a Machine learning | |
| 650 | 4 | |a Patient discharge | |
| 650 | 4 | |a Risk | |
| 650 | 4 | |a Suicide | |
| 700 | 1 | |a Yang, Xinyu |e verfasserin |4 aut | |
| 700 | 1 | |a Mullin, Brian |e verfasserin |4 aut | |
| 700 | 1 | |a Stettenbauer, Elizabeth |e verfasserin |4 aut | |
| 700 | 1 | |a Waddington, Marin |e verfasserin |4 aut | |
| 700 | 1 | |a Zhang, Alice |e verfasserin |4 aut | |
| 700 | 1 | |a Williams, Peyton |e verfasserin |4 aut | |
| 700 | 1 | |a Rios Perez, Gabriel E |e verfasserin |4 aut | |
| 700 | 1 | |a Cook, Benjamin Lê |e verfasserin |4 aut | |
| 773 | 0 | 8 | |i Enthalten in |t Journal of affective disorders |d 1989 |g 350(2024) vom: 01. März, Seite 382-387 |w (DE-627)NLM000465615 |x 1573-2517 |7 nnns |
| 773 | 1 | 8 | |g volume:350 |g year:2024 |g day:01 |g month:03 |g pages:382-387 |
| 856 | 4 | 0 | |u http://dx.doi.org/10.1016/j.jad.2023.12.059 |3 Volltext |
| 912 | |a GBV_USEFLAG_A | ||
| 912 | |a GBV_NLM | ||
| 951 | |a AR | ||
| 952 | |d 350 |j 2024 |b 01 |c 03 |h 382-387 | ||