Exploring Diagnostic Precision and Triage Proficiency : A Comparative Study of GPT-4 and Bard in Addressing Common Ophthalmic Complaints
In the modern era, patients often resort to the internet for answers to their health-related concerns, and clinics face challenges to providing timely response to patient concerns. This has led to a need to investigate the capabilities of AI chatbots for ophthalmic diagnosis and triage. In this in silico study, 80 simulated patient complaints in ophthalmology with varying urgency levels and clinical descriptors were entered into both ChatGPT and Bard in a systematic 3-step submission process asking chatbots to triage, diagnose, and evaluate urgency. Three ophthalmologists graded chatbot responses. Chatbots were significantly better at ophthalmic triage than diagnosis (90.0% appropriate triage vs. 48.8% correct leading diagnosis; p < 0.001), and GPT-4 performed better than Bard for appropriate triage recommendations (96.3% vs. 83.8%; p = 0.008), grader satisfaction for patient use (81.3% vs. 55.0%; p < 0.001), and lower potential harm rates (6.3% vs. 20.0%; p = 0.010). More descriptors improved the accuracy of diagnosis for both GPT-4 and Bard. These results indicate that chatbots may not need to recognize the correct diagnosis to provide appropriate ophthalmic triage, and there is a potential utility of these tools in aiding patients or triage staff; however, they are not a replacement for professional ophthalmic evaluation or advice.
| Medienart: |
E-Artikel |
|---|
| Erscheinungsjahr: |
2024 |
|---|---|
| Erschienen: |
2024 |
| Enthalten in: |
Zur Gesamtaufnahme - volume:11 |
|---|---|
| Enthalten in: |
Bioengineering (Basel, Switzerland) - 11(2024), 2 vom: 26. Jan. |
| Sprache: |
Englisch |
|---|
| Beteiligte Personen: |
Zandi, Roya [VerfasserIn] |
|---|
| Links: |
|---|
| Themen: |
Artificial intelligence |
|---|
| Anmerkungen: |
Date Revised 25.02.2024 published: Electronic Citation Status PubMed-not-MEDLINE |
|---|
| doi: |
10.3390/bioengineering11020120 |
|---|
| funding: |
|
|---|---|
| Förderinstitution / Projekttitel: |
|
| PPN (Katalog-ID): |
NLM368815897 |
|---|
| LEADER | 01000naa a22002652 4500 | ||
|---|---|---|---|
| 001 | NLM368815897 | ||
| 003 | DE-627 | ||
| 005 | 20240229151800.0 | ||
| 007 | cr uuu---uuuuu | ||
| 008 | 240229s2024 xx |||||o 00| ||eng c | ||
| 024 | 7 | |a 10.3390/bioengineering11020120 |2 doi | |
| 028 | 5 | 2 | |a pubmed24n1305.xml |
| 035 | |a (DE-627)NLM368815897 | ||
| 035 | |a (NLM)38391606 | ||
| 035 | |a (PII)120 | ||
| 040 | |a DE-627 |b ger |c DE-627 |e rakwb | ||
| 041 | |a eng | ||
| 100 | 1 | |a Zandi, Roya |e verfasserin |4 aut | |
| 245 | 1 | 0 | |a Exploring Diagnostic Precision and Triage Proficiency |b A Comparative Study of GPT-4 and Bard in Addressing Common Ophthalmic Complaints |
| 264 | 1 | |c 2024 | |
| 336 | |a Text |b txt |2 rdacontent | ||
| 337 | |a ƒaComputermedien |b c |2 rdamedia | ||
| 338 | |a ƒa Online-Ressource |b cr |2 rdacarrier | ||
| 500 | |a Date Revised 25.02.2024 | ||
| 500 | |a published: Electronic | ||
| 500 | |a Citation Status PubMed-not-MEDLINE | ||
| 520 | |a In the modern era, patients often resort to the internet for answers to their health-related concerns, and clinics face challenges to providing timely response to patient concerns. This has led to a need to investigate the capabilities of AI chatbots for ophthalmic diagnosis and triage. In this in silico study, 80 simulated patient complaints in ophthalmology with varying urgency levels and clinical descriptors were entered into both ChatGPT and Bard in a systematic 3-step submission process asking chatbots to triage, diagnose, and evaluate urgency. Three ophthalmologists graded chatbot responses. Chatbots were significantly better at ophthalmic triage than diagnosis (90.0% appropriate triage vs. 48.8% correct leading diagnosis; p < 0.001), and GPT-4 performed better than Bard for appropriate triage recommendations (96.3% vs. 83.8%; p = 0.008), grader satisfaction for patient use (81.3% vs. 55.0%; p < 0.001), and lower potential harm rates (6.3% vs. 20.0%; p = 0.010). More descriptors improved the accuracy of diagnosis for both GPT-4 and Bard. These results indicate that chatbots may not need to recognize the correct diagnosis to provide appropriate ophthalmic triage, and there is a potential utility of these tools in aiding patients or triage staff; however, they are not a replacement for professional ophthalmic evaluation or advice | ||
| 650 | 4 | |a Journal Article | |
| 650 | 4 | |a ChatGPT | |
| 650 | 4 | |a artificial intelligence | |
| 650 | 4 | |a bard | |
| 650 | 4 | |a chatbots | |
| 650 | 4 | |a large language models | |
| 650 | 4 | |a ophthalmology | |
| 650 | 4 | |a triage | |
| 700 | 1 | |a Fahey, Joseph D |e verfasserin |4 aut | |
| 700 | 1 | |a Drakopoulos, Michael |e verfasserin |4 aut | |
| 700 | 1 | |a Bryan, John M |e verfasserin |4 aut | |
| 700 | 1 | |a Dong, Siyuan |e verfasserin |4 aut | |
| 700 | 1 | |a Bryar, Paul J |e verfasserin |4 aut | |
| 700 | 1 | |a Bidwell, Ann E |e verfasserin |4 aut | |
| 700 | 1 | |a Bowen, R Chris |e verfasserin |4 aut | |
| 700 | 1 | |a Lavine, Jeremy A |e verfasserin |4 aut | |
| 700 | 1 | |a Mirza, Rukhsana G |e verfasserin |4 aut | |
| 773 | 0 | 8 | |i Enthalten in |t Bioengineering (Basel, Switzerland) |d 2014 |g 11(2024), 2 vom: 26. Jan. |w (DE-627)NLM259987786 |x 2306-5354 |7 nnns |
| 773 | 1 | 8 | |g volume:11 |g year:2024 |g number:2 |g day:26 |g month:01 |
| 856 | 4 | 0 | |u http://dx.doi.org/10.3390/bioengineering11020120 |3 Volltext |
| 912 | |a GBV_USEFLAG_A | ||
| 912 | |a GBV_NLM | ||
| 951 | |a AR | ||
| 952 | |d 11 |j 2024 |e 2 |b 26 |c 01 | ||