Bard Versus the 2022 American Society of Plastic Surgeons In-Service Examination : Performance on the Examination in Its Intern Year
© The Author(s) 2023. Published by Oxford University Press on behalf of The Aesthetic Society..
Background: Bard is a conversational generative artificial intelligence (AI) platform released by Google (Mountain View, CA) to the public in May 2023.
Objectives: This study investigates the performance of Bard on the American Society of Plastic Surgeons (ASPS) In-Service Examination to compare it to residents' performance nationally. We hypothesized that Bard would perform best on the comprehensive and core surgical principles portions of the examination.
Methods: Google's 2023 Bard was used to answer questions from the 2022 ASPS In-Service Examination. Each question was asked as written with the stem and multiple-choice options. The 2022 ASPS Norm Table was utilized to compare Bard's performance to that of subgroups of plastic surgery residents.
Results: A total of 231 questions were included. Bard answered 143 questions correctly corresponding to an accuracy of 62%. The highest-performing section was the comprehensive portion (73%). When compared with integrated residents nationally, Bard scored in the 74th percentile for post-graduate year (PGY)-1, 34th percentile for PGY-2, 20th percentile for PGY-3, 8th percentile for PGY-4, 1st percentile for PGY-5, and 2nd percentile for PGY-6.
Conclusions: Bard outperformed more than half of the first-year integrated residents (74th percentile). Its best sections were the comprehensive and core surgical principle portions of the examination. Further analysis of the chatbot's incorrect questions might help improve the overall quality of the examination's questions.
| Medienart: |
E-Artikel |
|---|
| Erscheinungsjahr: |
2024 |
|---|---|
| Erschienen: |
2024 |
| Enthalten in: |
Zur Gesamtaufnahme - volume:6 |
|---|---|
| Enthalten in: |
Aesthetic surgery journal. Open forum - 6(2024) vom: 03., Seite ojad066 |
| Sprache: |
Englisch |
|---|
| Beteiligte Personen: |
Najafali, Daniel [VerfasserIn] |
|---|
| Links: |
|---|
| Themen: |
|---|
| Anmerkungen: |
Date Revised 11.01.2024 published: Electronic-eCollection Citation Status PubMed-not-MEDLINE |
|---|
| doi: |
10.1093/asjof/ojad066 |
|---|
| funding: |
|
|---|---|
| Förderinstitution / Projekttitel: |
|
| PPN (Katalog-ID): |
NLM366875604 |
|---|
| LEADER | 01000naa a22002652 4500 | ||
|---|---|---|---|
| 001 | NLM366875604 | ||
| 003 | DE-627 | ||
| 005 | 20240114233947.0 | ||
| 007 | cr uuu---uuuuu | ||
| 008 | 240114s2024 xx |||||o 00| ||eng c | ||
| 024 | 7 | |a 10.1093/asjof/ojad066 |2 doi | |
| 028 | 5 | 2 | |a pubmed24n1256.xml |
| 035 | |a (DE-627)NLM366875604 | ||
| 035 | |a (NLM)38196964 | ||
| 040 | |a DE-627 |b ger |c DE-627 |e rakwb | ||
| 041 | |a eng | ||
| 100 | 1 | |a Najafali, Daniel |e verfasserin |4 aut | |
| 245 | 1 | 0 | |a Bard Versus the 2022 American Society of Plastic Surgeons In-Service Examination |b Performance on the Examination in Its Intern Year |
| 264 | 1 | |c 2024 | |
| 336 | |a Text |b txt |2 rdacontent | ||
| 337 | |a ƒaComputermedien |b c |2 rdamedia | ||
| 338 | |a ƒa Online-Ressource |b cr |2 rdacarrier | ||
| 500 | |a Date Revised 11.01.2024 | ||
| 500 | |a published: Electronic-eCollection | ||
| 500 | |a Citation Status PubMed-not-MEDLINE | ||
| 520 | |a © The Author(s) 2023. Published by Oxford University Press on behalf of The Aesthetic Society. | ||
| 520 | |a Background: Bard is a conversational generative artificial intelligence (AI) platform released by Google (Mountain View, CA) to the public in May 2023 | ||
| 520 | |a Objectives: This study investigates the performance of Bard on the American Society of Plastic Surgeons (ASPS) In-Service Examination to compare it to residents' performance nationally. We hypothesized that Bard would perform best on the comprehensive and core surgical principles portions of the examination | ||
| 520 | |a Methods: Google's 2023 Bard was used to answer questions from the 2022 ASPS In-Service Examination. Each question was asked as written with the stem and multiple-choice options. The 2022 ASPS Norm Table was utilized to compare Bard's performance to that of subgroups of plastic surgery residents | ||
| 520 | |a Results: A total of 231 questions were included. Bard answered 143 questions correctly corresponding to an accuracy of 62%. The highest-performing section was the comprehensive portion (73%). When compared with integrated residents nationally, Bard scored in the 74th percentile for post-graduate year (PGY)-1, 34th percentile for PGY-2, 20th percentile for PGY-3, 8th percentile for PGY-4, 1st percentile for PGY-5, and 2nd percentile for PGY-6 | ||
| 520 | |a Conclusions: Bard outperformed more than half of the first-year integrated residents (74th percentile). Its best sections were the comprehensive and core surgical principle portions of the examination. Further analysis of the chatbot's incorrect questions might help improve the overall quality of the examination's questions | ||
| 650 | 4 | |a Journal Article | |
| 700 | 1 | |a Reiche, Erik |e verfasserin |4 aut | |
| 700 | 1 | |a Araya, Sthefano |e verfasserin |4 aut | |
| 700 | 1 | |a Camacho, Justin M |e verfasserin |4 aut | |
| 700 | 1 | |a Liu, Farrah C |e verfasserin |4 aut | |
| 700 | 1 | |a Johnstone, Thomas |e verfasserin |4 aut | |
| 700 | 1 | |a Patel, Sameer A |e verfasserin |4 aut | |
| 700 | 1 | |a Morrison, Shane D |e verfasserin |4 aut | |
| 700 | 1 | |a Dorafshar, Amir H |e verfasserin |4 aut | |
| 700 | 1 | |a Fox, Paige M |e verfasserin |4 aut | |
| 773 | 0 | 8 | |i Enthalten in |t Aesthetic surgery journal. Open forum |d 2019 |g 6(2024) vom: 03., Seite ojad066 |w (DE-627)NLM323421407 |x 2631-4797 |7 nnns |
| 773 | 1 | 8 | |g volume:6 |g year:2024 |g day:03 |g pages:ojad066 |
| 856 | 4 | 0 | |u http://dx.doi.org/10.1093/asjof/ojad066 |3 Volltext |
| 912 | |a GBV_USEFLAG_A | ||
| 912 | |a GBV_NLM | ||
| 951 | |a AR | ||
| 952 | |d 6 |j 2024 |b 03 |h ojad066 | ||