Human intelligence versus Chat-GPT : who performs better in correctly classifying patients in triage?
Copyright © 2024 Elsevier Inc. All rights reserved..
INTRODUCTION: Chat-GPT is rapidly emerging as a promising and potentially revolutionary tool in medicine. One of its possible applications is the stratification of patients according to the severity of clinical conditions and prognosis during the triage evaluation in the emergency department (ED).
METHODS: Using a randomly selected sample of 30 vignettes recreated from real clinical cases, we compared the concordance in risk stratification of ED patients between healthcare personnel and Chat-GPT. The concordance was assessed with Cohen's kappa, and the performance was evaluated with the area under the receiver operating characteristic curve (AUROC) curves. Among the outcomes, we considered mortality within 72 h, the need for hospitalization, and the presence of a severe or time-dependent condition.
RESULTS: The concordance in triage code assignment between triage nurses and Chat-GPT was 0.278 (unweighted Cohen's kappa; 95% confidence intervals: 0.231-0.388). For all outcomes, the ROC values were higher for the triage nurses. The most relevant difference was found in 72-h mortality, where triage nurses showed an AUROC of 0.910 (0.757-1.000) compared to only 0.669 (0.153-1.000) for Chat-GPT.
CONCLUSIONS: The current level of Chat-GPT reliability is insufficient to make it a valid substitute for the expertise of triage nurses in prioritizing ED patients. Further developments are required to enhance the safety and effectiveness of AI for risk stratification of ED patients.
| Medienart: |
E-Artikel |
|---|
| Erscheinungsjahr: |
2024 |
|---|---|
| Erschienen: |
2024 |
| Enthalten in: |
Zur Gesamtaufnahme - volume:79 |
|---|---|
| Enthalten in: |
The American journal of emergency medicine - 79(2024) vom: 09. Apr., Seite 44-47 |
| Sprache: |
Englisch |
|---|
| Beteiligte Personen: |
Zaboli, Arian [VerfasserIn] |
|---|
| Links: |
|---|
| Themen: |
Advanced nurse practice |
|---|
| Anmerkungen: |
Date Completed 16.04.2024 Date Revised 16.04.2024 published: Print-Electronic Citation Status MEDLINE |
|---|
| doi: |
10.1016/j.ajem.2024.02.008 |
|---|
| funding: |
|
|---|---|
| Förderinstitution / Projekttitel: |
|
| PPN (Katalog-ID): |
NLM368315576 |
|---|
| LEADER | 01000caa a22002652 4500 | ||
|---|---|---|---|
| 001 | NLM368315576 | ||
| 003 | DE-627 | ||
| 005 | 20240416232532.0 | ||
| 007 | cr uuu---uuuuu | ||
| 008 | 240212s2024 xx |||||o 00| ||eng c | ||
| 024 | 7 | |a 10.1016/j.ajem.2024.02.008 |2 doi | |
| 028 | 5 | 2 | |a pubmed24n1377.xml |
| 035 | |a (DE-627)NLM368315576 | ||
| 035 | |a (NLM)38341993 | ||
| 035 | |a (PII)S0735-6757(24)00066-4 | ||
| 040 | |a DE-627 |b ger |c DE-627 |e rakwb | ||
| 041 | |a eng | ||
| 100 | 1 | |a Zaboli, Arian |e verfasserin |4 aut | |
| 245 | 1 | 0 | |a Human intelligence versus Chat-GPT |b who performs better in correctly classifying patients in triage? |
| 264 | 1 | |c 2024 | |
| 336 | |a Text |b txt |2 rdacontent | ||
| 337 | |a ƒaComputermedien |b c |2 rdamedia | ||
| 338 | |a ƒa Online-Ressource |b cr |2 rdacarrier | ||
| 500 | |a Date Completed 16.04.2024 | ||
| 500 | |a Date Revised 16.04.2024 | ||
| 500 | |a published: Print-Electronic | ||
| 500 | |a Citation Status MEDLINE | ||
| 520 | |a Copyright © 2024 Elsevier Inc. All rights reserved. | ||
| 520 | |a INTRODUCTION: Chat-GPT is rapidly emerging as a promising and potentially revolutionary tool in medicine. One of its possible applications is the stratification of patients according to the severity of clinical conditions and prognosis during the triage evaluation in the emergency department (ED) | ||
| 520 | |a METHODS: Using a randomly selected sample of 30 vignettes recreated from real clinical cases, we compared the concordance in risk stratification of ED patients between healthcare personnel and Chat-GPT. The concordance was assessed with Cohen's kappa, and the performance was evaluated with the area under the receiver operating characteristic curve (AUROC) curves. Among the outcomes, we considered mortality within 72 h, the need for hospitalization, and the presence of a severe or time-dependent condition | ||
| 520 | |a RESULTS: The concordance in triage code assignment between triage nurses and Chat-GPT was 0.278 (unweighted Cohen's kappa; 95% confidence intervals: 0.231-0.388). For all outcomes, the ROC values were higher for the triage nurses. The most relevant difference was found in 72-h mortality, where triage nurses showed an AUROC of 0.910 (0.757-1.000) compared to only 0.669 (0.153-1.000) for Chat-GPT | ||
| 520 | |a CONCLUSIONS: The current level of Chat-GPT reliability is insufficient to make it a valid substitute for the expertise of triage nurses in prioritizing ED patients. Further developments are required to enhance the safety and effectiveness of AI for risk stratification of ED patients | ||
| 650 | 4 | |a Journal Article | |
| 650 | 4 | |a Advanced nurse practice | |
| 650 | 4 | |a Artificial intelligence | |
| 650 | 4 | |a ChatGPT | |
| 650 | 4 | |a Manchester triage system | |
| 650 | 4 | |a Nursing | |
| 650 | 4 | |a Triage | |
| 700 | 1 | |a Brigo, Francesco |e verfasserin |4 aut | |
| 700 | 1 | |a Sibilio, Serena |e verfasserin |4 aut | |
| 700 | 1 | |a Mian, Michael |e verfasserin |4 aut | |
| 700 | 1 | |a Turcato, Gianni |e verfasserin |4 aut | |
| 773 | 0 | 8 | |i Enthalten in |t The American journal of emergency medicine |d 1986 |g 79(2024) vom: 09. Apr., Seite 44-47 |w (DE-627)NLM01297868X |x 1532-8171 |7 nnns |
| 773 | 1 | 8 | |g volume:79 |g year:2024 |g day:09 |g month:04 |g pages:44-47 |
| 856 | 4 | 0 | |u http://dx.doi.org/10.1016/j.ajem.2024.02.008 |3 Volltext |
| 912 | |a GBV_USEFLAG_A | ||
| 912 | |a GBV_NLM | ||
| 951 | |a AR | ||
| 952 | |d 79 |j 2024 |b 09 |c 04 |h 44-47 | ||