Holistic Evaluation of GPT-4V for Biomedical Imaging
In this paper, we present a large-scale evaluation probing GPT-4V's capabilities and limitations for biomedical image analysis. GPT-4V represents a breakthrough in artificial general intelligence (AGI) for computer vision, with applications in the biomedical domain. We assess GPT-4V's performance across 16 medical imaging categories, including radiology, oncology, ophthalmology, pathology, and more. Tasks include modality recognition, anatomy localization, disease diagnosis, report generation, and lesion detection. The extensive experiments provide insights into GPT-4V's strengths and weaknesses. Results show GPT-4V's proficiency in modality and anatomy recognition but difficulty with disease diagnosis and localization. GPT-4V excels at diagnostic report generation, indicating strong image captioning skills. While promising for biomedical imaging AI, GPT-4V requires further enhancement and validation before clinical deployment. We emphasize responsible development and testing for trustworthy integration of biomedical AGI. This rigorous evaluation of GPT-4V on diverse medical images advances understanding of multimodal large language models (LLMs) and guides future work toward impactful healthcare applications..
| Medienart: |
|
|---|
Preprint| Erscheinungsjahr: |
2023 |
|---|---|
| Erschienen: |
2023 |
| Enthalten in: |
arXiv.org - (2023) vom: 10. Nov. Zur Gesamtaufnahme - year:2023 |
|---|
| Sprache: |
Englisch |
|---|
| Beteiligte Personen: |
Liu, Zhengliang [VerfasserIn] |
|---|
| Links: |
Volltext [kostenfrei] |
|---|
| Themen: |
000 |
|---|
| Förderinstitution / Projekttitel: |
|
|---|
| PPN (Katalog-ID): |
XAR041819241 |
|---|
| LEADER | 01000naa a22002652 4500 | ||
|---|---|---|---|
| 001 | XAR041819241 | ||
| 003 | DE-627 | ||
| 005 | 20231212080316.0 | ||
| 007 | cr uuu---uuuuu | ||
| 008 | 231212s2023 xx |||||o 00| ||eng c | ||
| 035 | |a (DE-627)XAR041819241 | ||
| 035 | |a (arXiv)2312.05256 | ||
| 040 | |a DE-627 |b ger |c DE-627 |e rakwb | ||
| 041 | |a eng | ||
| 100 | 1 | |a Liu, Zhengliang |e verfasserin |4 aut | |
| 245 | 1 | 0 | |a Holistic Evaluation of GPT-4V for Biomedical Imaging |
| 264 | 1 | |c 2023 | |
| 336 | |a Text |b txt |2 rdacontent | ||
| 337 | |a Computermedien |b c |2 rdamedia | ||
| 338 | |a Online-Ressource |b cr |2 rdacarrier | ||
| 520 | |a In this paper, we present a large-scale evaluation probing GPT-4V's capabilities and limitations for biomedical image analysis. GPT-4V represents a breakthrough in artificial general intelligence (AGI) for computer vision, with applications in the biomedical domain. We assess GPT-4V's performance across 16 medical imaging categories, including radiology, oncology, ophthalmology, pathology, and more. Tasks include modality recognition, anatomy localization, disease diagnosis, report generation, and lesion detection. The extensive experiments provide insights into GPT-4V's strengths and weaknesses. Results show GPT-4V's proficiency in modality and anatomy recognition but difficulty with disease diagnosis and localization. GPT-4V excels at diagnostic report generation, indicating strong image captioning skills. While promising for biomedical imaging AI, GPT-4V requires further enhancement and validation before clinical deployment. We emphasize responsible development and testing for trustworthy integration of biomedical AGI. This rigorous evaluation of GPT-4V on diverse medical images advances understanding of multimodal large language models (LLMs) and guides future work toward impactful healthcare applications. | ||
| 650 | 4 | |a Electrical Engineering and Systems Science - Image and Video Processing |7 (dpeaa)DE-84 | |
| 650 | 4 | |a Computer Science - Artificial Intelligence |7 (dpeaa)DE-84 | |
| 650 | 4 | |a 620 |7 (dpeaa)DE-84 | |
| 650 | 4 | |a 000 |7 (dpeaa)DE-84 | |
| 700 | 1 | |a Jiang, Hanqi |4 aut | |
| 700 | 1 | |a Zhong, Tianyang |4 aut | |
| 700 | 1 | |a Wu, Zihao |4 aut | |
| 700 | 1 | |a Ma, Chong |4 aut | |
| 700 | 1 | |a Li, Yiwei |4 aut | |
| 700 | 1 | |a Yu, Xiaowei |4 aut | |
| 700 | 1 | |a Zhang, Yutong |4 aut | |
| 700 | 1 | |a Pan, Yi |4 aut | |
| 700 | 1 | |a Shu, Peng |4 aut | |
| 700 | 1 | |a Lyu, Yanjun |4 aut | |
| 700 | 1 | |a Zhang, Lu |4 aut | |
| 700 | 1 | |a Yao, Junjie |4 aut | |
| 700 | 1 | |a Dong, Peixin |4 aut | |
| 700 | 1 | |a Cao, Chao |4 aut | |
| 700 | 1 | |a Xiao, Zhenxiang |4 aut | |
| 700 | 1 | |a Wang, Jiaqi |4 aut | |
| 700 | 1 | |a Zhao, Huan |4 aut | |
| 700 | 1 | |a Xu, Shaochen |4 aut | |
| 700 | 1 | |a Wei, Yaonai |4 aut | |
| 700 | 1 | |a Chen, Jingyuan |4 aut | |
| 700 | 1 | |a Dai, Haixing |4 aut | |
| 700 | 1 | |a Wang, Peilong |4 aut | |
| 700 | 1 | |a He, Hao |4 aut | |
| 700 | 1 | |a Wang, Zewei |4 aut | |
| 700 | 1 | |a Wang, Xinyu |4 aut | |
| 700 | 1 | |a Zhang, Xu |4 aut | |
| 700 | 1 | |a Zhao, Lin |4 aut | |
| 700 | 1 | |a Liu, Yiheng |4 aut | |
| 700 | 1 | |a Zhang, Kai |4 aut | |
| 700 | 1 | |a Yan, Liheng |4 aut | |
| 700 | 1 | |a Sun, Lichao |4 aut | |
| 700 | 1 | |a Liu, Jun |4 aut | |
| 700 | 1 | |a Qiang, Ning |4 aut | |
| 700 | 1 | |a Ge, Bao |4 aut | |
| 700 | 1 | |a Cai, Xiaoyan |4 aut | |
| 700 | 1 | |a Zhao, Shijie |4 aut | |
| 700 | 1 | |a Hu, Xintao |4 aut | |
| 700 | 1 | |a Yuan, Yixuan |4 aut | |
| 700 | 1 | |a Li, Gang |4 aut | |
| 700 | 1 | |a Zhang, Shu |4 aut | |
| 700 | 1 | |a Zhang, Xin |4 aut | |
| 700 | 1 | |a Jiang, Xi |4 aut | |
| 700 | 1 | |a Zhang, Tuo |4 aut | |
| 700 | 1 | |a Shen, Dinggang |4 aut | |
| 700 | 1 | |a Li, Quanzheng |4 aut | |
| 700 | 1 | |a Liu, Wei |4 aut | |
| 700 | 1 | |a Li, Xiang |4 aut | |
| 700 | 1 | |a Zhu, Dajiang |4 aut | |
| 700 | 1 | |a Liu, Tianming |4 aut | |
| 773 | 0 | 8 | |i Enthalten in |t arXiv.org |g (2023) vom: 10. Nov. |
| 773 | 1 | 8 | |g year:2023 |g day:10 |g month:11 |
| 856 | 4 | 0 | |u https://arxiv.org/abs/2312.05256 |z kostenfrei |3 Volltext |
| 912 | |a GBV_XAR | ||
| 951 | |a AR | ||
| 952 | |j 2023 |b 10 |c 11 | ||