Development and validation of a deep learning model for multicategory pneumonia classification on chest computed tomography : a multicenter and multireader study
2023 Quantitative Imaging in Medicine and Surgery. All rights reserved..
Background: Accurate diagnosis of pneumonia is vital for effective disease management and mortality reduction, but it can be easily confused with other conditions on chest computed tomography (CT) due to an overlap in imaging features. We aimed to develop and validate a deep learning (DL) model based on chest CT for accurate classification of viral pneumonia (VP), bacterial pneumonia (BP), fungal pneumonia (FP), pulmonary tuberculosis (PTB), and no pneumonia (NP) conditions.
Methods: In total, 1,776 cases from five hospitals in different regions were retrospectively collected from September 2019 to June 2023. All cases were enrolled according to inclusion and exclusion criteria, and ultimately 1,611 cases were used to develop the DL model with 5-fold cross-validation, with 165 cases being used as the external test set. Five radiologists blindly reviewed the images from the internal and external test sets first without and then with DL model assistance. Precision, recall, F1-score, weighted F1-average, and area under the curve (AUC) were used to evaluate the model performance.
Results: The F1-scores of the DL model on the internal and external test sets were, respectively, 0.947 [95% confidence interval (CI): 0.936-0.958] and 0.933 (95% CI: 0.916-0.950) for VP, 0.511 (95% CI: 0.487-0.536) and 0.591 (95% CI: 0.557-0.624) for BP, 0.842 (95% CI: 0.824-0.860) and 0.848 (95% CI: 0.824-0.873) for FP, 0.843 (95% CI: 0.826-0.861) and 0.795 (95% CI: 0.767-0.822) for PTB, and 0.975 (95% CI: 0.968-0.983) and 0.976 (95% CI: 0.965-0.986) for NP, with a weighted F1-average of 0.883 (95% CI: 0.867-0.898) and 0.846 (95% CI: 0.822-0.871), respectively. The model performed well and showed comparable performance in both the internal and external test sets. The F1-score of the DL model was higher than that of radiologists, and with DL model assistance, radiologists achieved a higher F1-score. On the external test set, the F1-score of the DL model (F1-score 0.848; 95% CI: 0.824-0.873) was higher than that of the radiologists (F1-score 0.541; 95% CI: 0.507-0.575) as was its precision for the other three pneumonia conditions (all P values <0.001). With DL model assistance, the F1-score for FP (F1-score 0.541; 95% CI: 0.507-0.575) was higher than that achieved without assistance (F1-score 0.778; 95% CI: 0.750-0.807) as was its precision for the other three pneumonia conditions (all P values <0.001).
Conclusions: The DL approach can effectively classify pneumonia and can help improve radiologists' performance, supporting the full integration of DL results into the routine workflow of clinicians.
Medienart: |
E-Artikel |
---|
Erscheinungsjahr: |
2023 |
---|---|
Erschienen: |
2023 |
Enthalten in: |
Zur Gesamtaufnahme - volume:13 |
---|---|
Enthalten in: |
Quantitative imaging in medicine and surgery - 13(2023), 12 vom: 01. Dez., Seite 8641-8656 |
Sprache: |
Englisch |
---|
Beteiligte Personen: |
Shi, Chunzi [VerfasserIn] |
---|
Links: |
---|
Themen: |
Computed tomography (CT) |
---|
Anmerkungen: |
Date Revised 19.12.2023 published: Print-Electronic Citation Status PubMed-not-MEDLINE |
---|
doi: |
10.21037/qims-23-1097 |
---|
funding: |
|
---|---|
Förderinstitution / Projekttitel: |
|
PPN (Katalog-ID): |
NLM36596963X |
---|
LEADER | 01000naa a22002652 4500 | ||
---|---|---|---|
001 | NLM36596963X | ||
003 | DE-627 | ||
005 | 20231227134528.0 | ||
007 | cr uuu---uuuuu | ||
008 | 231227s2023 xx |||||o 00| ||eng c | ||
024 | 7 | |a 10.21037/qims-23-1097 |2 doi | |
028 | 5 | 2 | |a pubmed24n1232.xml |
035 | |a (DE-627)NLM36596963X | ||
035 | |a (NLM)38106268 | ||
040 | |a DE-627 |b ger |c DE-627 |e rakwb | ||
041 | |a eng | ||
100 | 1 | |a Shi, Chunzi |e verfasserin |4 aut | |
245 | 1 | 0 | |a Development and validation of a deep learning model for multicategory pneumonia classification on chest computed tomography |b a multicenter and multireader study |
264 | 1 | |c 2023 | |
336 | |a Text |b txt |2 rdacontent | ||
337 | |a ƒaComputermedien |b c |2 rdamedia | ||
338 | |a ƒa Online-Ressource |b cr |2 rdacarrier | ||
500 | |a Date Revised 19.12.2023 | ||
500 | |a published: Print-Electronic | ||
500 | |a Citation Status PubMed-not-MEDLINE | ||
520 | |a 2023 Quantitative Imaging in Medicine and Surgery. All rights reserved. | ||
520 | |a Background: Accurate diagnosis of pneumonia is vital for effective disease management and mortality reduction, but it can be easily confused with other conditions on chest computed tomography (CT) due to an overlap in imaging features. We aimed to develop and validate a deep learning (DL) model based on chest CT for accurate classification of viral pneumonia (VP), bacterial pneumonia (BP), fungal pneumonia (FP), pulmonary tuberculosis (PTB), and no pneumonia (NP) conditions | ||
520 | |a Methods: In total, 1,776 cases from five hospitals in different regions were retrospectively collected from September 2019 to June 2023. All cases were enrolled according to inclusion and exclusion criteria, and ultimately 1,611 cases were used to develop the DL model with 5-fold cross-validation, with 165 cases being used as the external test set. Five radiologists blindly reviewed the images from the internal and external test sets first without and then with DL model assistance. Precision, recall, F1-score, weighted F1-average, and area under the curve (AUC) were used to evaluate the model performance | ||
520 | |a Results: The F1-scores of the DL model on the internal and external test sets were, respectively, 0.947 [95% confidence interval (CI): 0.936-0.958] and 0.933 (95% CI: 0.916-0.950) for VP, 0.511 (95% CI: 0.487-0.536) and 0.591 (95% CI: 0.557-0.624) for BP, 0.842 (95% CI: 0.824-0.860) and 0.848 (95% CI: 0.824-0.873) for FP, 0.843 (95% CI: 0.826-0.861) and 0.795 (95% CI: 0.767-0.822) for PTB, and 0.975 (95% CI: 0.968-0.983) and 0.976 (95% CI: 0.965-0.986) for NP, with a weighted F1-average of 0.883 (95% CI: 0.867-0.898) and 0.846 (95% CI: 0.822-0.871), respectively. The model performed well and showed comparable performance in both the internal and external test sets. The F1-score of the DL model was higher than that of radiologists, and with DL model assistance, radiologists achieved a higher F1-score. On the external test set, the F1-score of the DL model (F1-score 0.848; 95% CI: 0.824-0.873) was higher than that of the radiologists (F1-score 0.541; 95% CI: 0.507-0.575) as was its precision for the other three pneumonia conditions (all P values <0.001). With DL model assistance, the F1-score for FP (F1-score 0.541; 95% CI: 0.507-0.575) was higher than that achieved without assistance (F1-score 0.778; 95% CI: 0.750-0.807) as was its precision for the other three pneumonia conditions (all P values <0.001) | ||
520 | |a Conclusions: The DL approach can effectively classify pneumonia and can help improve radiologists' performance, supporting the full integration of DL results into the routine workflow of clinicians | ||
650 | 4 | |a Journal Article | |
650 | 4 | |a Deep learning (DL) | |
650 | 4 | |a computed tomography (CT) | |
650 | 4 | |a pneumonia | |
650 | 4 | |a prediction | |
700 | 1 | |a Shao, Ying |e verfasserin |4 aut | |
700 | 1 | |a Shan, Fei |e verfasserin |4 aut | |
700 | 1 | |a Shen, Jie |e verfasserin |4 aut | |
700 | 1 | |a Huang, Xueni |e verfasserin |4 aut | |
700 | 1 | |a Chen, Chuan |e verfasserin |4 aut | |
700 | 1 | |a Lu, Yang |e verfasserin |4 aut | |
700 | 1 | |a Zhan, Yi |e verfasserin |4 aut | |
700 | 1 | |a Shi, Nannan |e verfasserin |4 aut | |
700 | 1 | |a Wu, Jili |e verfasserin |4 aut | |
700 | 1 | |a Wang, Keying |e verfasserin |4 aut | |
700 | 1 | |a Gao, Yaozong |e verfasserin |4 aut | |
700 | 1 | |a Shi, Yuxin |e verfasserin |4 aut | |
700 | 1 | |a Song, Fengxiang |e verfasserin |4 aut | |
773 | 0 | 8 | |i Enthalten in |t Quantitative imaging in medicine and surgery |d 2011 |g 13(2023), 12 vom: 01. Dez., Seite 8641-8656 |w (DE-627)NLM216114624 |x 2223-4292 |7 nnns |
773 | 1 | 8 | |g volume:13 |g year:2023 |g number:12 |g day:01 |g month:12 |g pages:8641-8656 |
856 | 4 | 0 | |u http://dx.doi.org/10.21037/qims-23-1097 |3 Volltext |
912 | |a GBV_USEFLAG_A | ||
912 | |a GBV_NLM | ||
951 | |a AR | ||
952 | |d 13 |j 2023 |e 12 |b 01 |c 12 |h 8641-8656 |