Development and validation of an early warning model for hospitalized COVID-19 patients: a multi-center retrospective cohort study
Background Timely identification of deteriorating COVID-19 patients is needed to guide changes in clinical management and admission to intensive care units (ICUs). There is significant concern that widely used Early warning scores (EWSs) underestimate illness severity in COVID-19 patients and therefore, we developed an early warning model specifically for COVID-19 patients. Methods We retrospectively collected electronic medical record data to extract predictors and used these to fit a random forest model. To simulate the situation in which the model would have been developed after the first and implemented during the second COVID-19 ‘wave’ in the Netherlands, we performed a temporal validation by splitting all included patients into groups admitted before and after August 1, 2020. Furthermore, we propose a method for dynamic model updating to retain model performance over time. We evaluated model discrimination and calibration, performed a decision curve analysis, and quantified the importance of predictors using SHapley Additive exPlanations values. Results We included 3514 COVID-19 patient admissions from six Dutch hospitals between February 2020 and May 2021, and included a total of 18 predictors for model fitting. The model showed a higher discriminative performance in terms of partial area under the receiver operating characteristic curve (0.82 [0.80–0.84]) compared to the National early warning score (0.72 [0.69–0.74]) and the Modified early warning score (0.67 [0.65–0.69]), a greater net benefit over a range of clinically relevant model thresholds, and relatively good calibration (intercept = 0.03 [− 0.09 to 0.14], slope = 0.79 [0.73–0.86]). Conclusions This study shows the potential benefit of moving from early warning models for the general inpatient population to models for specific patient groups. Further (independent) validation of the model is needed..
| Medienart: |
E-Artikel |
|---|
| Erscheinungsjahr: |
2022 |
|---|---|
| Erschienen: |
2022 |
| Enthalten in: |
Zur Gesamtaufnahme - volume:10 |
|---|---|
| Enthalten in: |
Intensive Care Medicine Experimental - 10(2022), 1 vom: 19. Sept. |
| Sprache: |
Englisch |
|---|
| Beteiligte Personen: |
Smit, Jim M. [VerfasserIn] |
|---|
| Links: |
Volltext [kostenfrei] |
|---|
| Themen: |
Artificial intelligence |
|---|
| Anmerkungen: |
© The Author(s) 2022 |
|---|
| doi: |
10.1186/s40635-022-00465-4 |
|---|
| funding: |
|
|---|---|
| Förderinstitution / Projekttitel: |
|
| PPN (Katalog-ID): |
OLC2132080763 |
|---|
| LEADER | 01000caa a22002652 4500 | ||
|---|---|---|---|
| 001 | OLC2132080763 | ||
| 003 | DE-627 | ||
| 005 | 20240410143536.0 | ||
| 007 | cr uuu---uuuuu | ||
| 008 | 230506s2022 xx |||||o 00| ||eng c | ||
| 024 | 7 | |a 10.1186/s40635-022-00465-4 |2 doi | |
| 035 | |a (DE-627)OLC2132080763 | ||
| 035 | |a (DE-He213)s40635-022-00465-4-e | ||
| 040 | |a DE-627 |b ger |c DE-627 |e rakwb | ||
| 041 | |a eng | ||
| 082 | 0 | 4 | |a 610 |q VZ |
| 100 | 1 | |a Smit, Jim M. |e verfasserin |0 (orcid)0000-0003-3716-1863 |4 aut | |
| 245 | 1 | 0 | |a Development and validation of an early warning model for hospitalized COVID-19 patients: a multi-center retrospective cohort study |
| 264 | 1 | |c 2022 | |
| 336 | |a Text |b txt |2 rdacontent | ||
| 337 | |a Computermedien |b c |2 rdamedia | ||
| 338 | |a Online-Ressource |b cr |2 rdacarrier | ||
| 500 | |a © The Author(s) 2022 | ||
| 520 | |a Background Timely identification of deteriorating COVID-19 patients is needed to guide changes in clinical management and admission to intensive care units (ICUs). There is significant concern that widely used Early warning scores (EWSs) underestimate illness severity in COVID-19 patients and therefore, we developed an early warning model specifically for COVID-19 patients. Methods We retrospectively collected electronic medical record data to extract predictors and used these to fit a random forest model. To simulate the situation in which the model would have been developed after the first and implemented during the second COVID-19 ‘wave’ in the Netherlands, we performed a temporal validation by splitting all included patients into groups admitted before and after August 1, 2020. Furthermore, we propose a method for dynamic model updating to retain model performance over time. We evaluated model discrimination and calibration, performed a decision curve analysis, and quantified the importance of predictors using SHapley Additive exPlanations values. Results We included 3514 COVID-19 patient admissions from six Dutch hospitals between February 2020 and May 2021, and included a total of 18 predictors for model fitting. The model showed a higher discriminative performance in terms of partial area under the receiver operating characteristic curve (0.82 [0.80–0.84]) compared to the National early warning score (0.72 [0.69–0.74]) and the Modified early warning score (0.67 [0.65–0.69]), a greater net benefit over a range of clinically relevant model thresholds, and relatively good calibration (intercept = 0.03 [− 0.09 to 0.14], slope = 0.79 [0.73–0.86]). Conclusions This study shows the potential benefit of moving from early warning models for the general inpatient population to models for specific patient groups. Further (independent) validation of the model is needed. | ||
| 650 | 4 | |a COVID-19 | |
| 650 | 4 | |a Early warning score | |
| 650 | 4 | |a Intensive care | |
| 650 | 4 | |a Machine learning | |
| 650 | 4 | |a Artificial intelligence | |
| 650 | 4 | |a Medical prediction model | |
| 650 | 4 | |a Dynamic model updating | |
| 700 | 1 | |a Krijthe, Jesse H. |4 aut | |
| 700 | 1 | |a Tintu, Andrei N. |4 aut | |
| 700 | 1 | |a Endeman, Henrik |4 aut | |
| 700 | 1 | |a Ludikhuize, Jeroen |4 aut | |
| 700 | 1 | |a van Genderen, Michel E. |4 aut | |
| 700 | 1 | |a Hassan, Shermarke |4 aut | |
| 700 | 1 | |a El Moussaoui, Rachida |4 aut | |
| 700 | 1 | |a Westerweel, Peter E. |4 aut | |
| 700 | 1 | |a Goekoop, Robbert J. |4 aut | |
| 700 | 1 | |a Waverijn, Geeke |4 aut | |
| 700 | 1 | |a Verheijen, Tim |4 aut | |
| 700 | 1 | |a den Hollander, Jan G. |4 aut | |
| 700 | 1 | |a de Boer, Mark G. J. |4 aut | |
| 700 | 1 | |a Gommers, Diederik A. M. P. J. |4 aut | |
| 700 | 1 | |a van der Vlies, Robin |4 aut | |
| 700 | 1 | |a Schellings, Mark |4 aut | |
| 700 | 1 | |a Carels, Regina A. |4 aut | |
| 700 | 1 | |a van Nieuwkoop, Cees |4 aut | |
| 700 | 1 | |a Arbous, Sesmu M. |4 aut | |
| 700 | 1 | |a van Bommel, Jasper |4 aut | |
| 700 | 1 | |a Knevel, Rachel |4 aut | |
| 700 | 1 | |a de Rijke, Yolanda B. |4 aut | |
| 700 | 1 | |a Reinders, Marcel J. T. |4 aut | |
| 773 | 0 | 8 | |i Enthalten in |t Intensive Care Medicine Experimental |d Springer International Publishing, 2013 |g 10(2022), 1 vom: 19. Sept. |h Online-Ressource |w (DE-627)771394640 |w (DE-600)2740385-3 |w (DE-576)396696023 |x 2197-425X |7 nnns |
| 773 | 1 | 8 | |g volume:10 |g year:2022 |g number:1 |g day:19 |g month:09 |
| 856 | 4 | 0 | |u https://dx.doi.org/10.1186/s40635-022-00465-4 |z kostenfrei |3 Volltext |
| 912 | |a GBV_USEFLAG_A | ||
| 912 | |a SYSFLAG_A | ||
| 912 | |a GBV_OLC | ||
| 912 | |a SSG-OLC-PHA | ||
| 912 | |a GBV_ILN_20 | ||
| 912 | |a GBV_ILN_22 | ||
| 912 | |a GBV_ILN_23 | ||
| 912 | |a GBV_ILN_24 | ||
| 912 | |a GBV_ILN_31 | ||
| 912 | |a GBV_ILN_39 | ||
| 912 | |a GBV_ILN_40 | ||
| 912 | |a GBV_ILN_60 | ||
| 912 | |a GBV_ILN_62 | ||
| 912 | |a GBV_ILN_63 | ||
| 912 | |a GBV_ILN_65 | ||
| 912 | |a GBV_ILN_69 | ||
| 912 | |a GBV_ILN_73 | ||
| 912 | |a GBV_ILN_74 | ||
| 912 | |a GBV_ILN_95 | ||
| 912 | |a GBV_ILN_105 | ||
| 912 | |a GBV_ILN_110 | ||
| 912 | |a GBV_ILN_151 | ||
| 912 | |a GBV_ILN_161 | ||
| 912 | |a GBV_ILN_170 | ||
| 912 | |a GBV_ILN_206 | ||
| 912 | |a GBV_ILN_213 | ||
| 912 | |a GBV_ILN_230 | ||
| 912 | |a GBV_ILN_285 | ||
| 912 | |a GBV_ILN_293 | ||
| 912 | |a GBV_ILN_602 | ||
| 912 | |a GBV_ILN_2005 | ||
| 912 | |a GBV_ILN_2009 | ||
| 912 | |a GBV_ILN_2014 | ||
| 912 | |a GBV_ILN_2055 | ||
| 912 | |a GBV_ILN_2111 | ||
| 912 | |a GBV_ILN_2446 | ||
| 912 | |a GBV_ILN_4012 | ||
| 912 | |a GBV_ILN_4037 | ||
| 912 | |a GBV_ILN_4112 | ||
| 912 | |a GBV_ILN_4125 | ||
| 912 | |a GBV_ILN_4126 | ||
| 912 | |a GBV_ILN_4249 | ||
| 912 | |a GBV_ILN_4305 | ||
| 912 | |a GBV_ILN_4306 | ||
| 912 | |a GBV_ILN_4307 | ||
| 912 | |a GBV_ILN_4313 | ||
| 912 | |a GBV_ILN_4322 | ||
| 912 | |a GBV_ILN_4323 | ||
| 912 | |a GBV_ILN_4324 | ||
| 912 | |a GBV_ILN_4325 | ||
| 912 | |a GBV_ILN_4338 | ||
| 912 | |a GBV_ILN_4367 | ||
| 912 | |a GBV_ILN_4700 | ||
| 951 | |a AR | ||
| 952 | |d 10 |j 2022 |e 1 |b 19 |c 09 | ||