Federated Learning for Decentralized Artificial Intelligence in Melanoma Diagnostics
Importance: The development of artificial intelligence (AI)-based melanoma classifiers typically calls for large, centralized datasets, requiring hospitals to give away their patient data, which raises serious privacy concerns. To address this concern, decentralized federated learning has been proposed, where classifier development is distributed across hospitals.
Objective: To investigate whether a more privacy-preserving federated learning approach can achieve comparable diagnostic performance to a classical centralized (ie, single-model) and ensemble learning approach for AI-based melanoma diagnostics.
Design, Setting, and Participants: This multicentric, single-arm diagnostic study developed a federated model for melanoma-nevus classification using histopathological whole-slide images prospectively acquired at 6 German university hospitals between April 2021 and February 2023 and benchmarked it using both a holdout and an external test dataset. Data analysis was performed from February to April 2023.
Exposures: All whole-slide images were retrospectively analyzed by an AI-based classifier without influencing routine clinical care.
Main Outcomes and Measures: The area under the receiver operating characteristic curve (AUROC) served as the primary end point for evaluating the diagnostic performance. Secondary end points included balanced accuracy, sensitivity, and specificity.
Results: The study included 1025 whole-slide images of clinically melanoma-suspicious skin lesions from 923 patients, consisting of 388 histopathologically confirmed invasive melanomas and 637 nevi. The median (range) age at diagnosis was 58 (18-95) years for the training set, 57 (18-93) years for the holdout test dataset, and 61 (18-95) years for the external test dataset; the median (range) Breslow thickness was 0.70 (0.10-34.00) mm, 0.70 (0.20-14.40) mm, and 0.80 (0.30-20.00) mm, respectively. The federated approach (0.8579; 95% CI, 0.7693-0.9299) performed significantly worse than the classical centralized approach (0.9024; 95% CI, 0.8379-0.9565) in terms of AUROC on a holdout test dataset (pairwise Wilcoxon signed-rank, P < .001) but performed significantly better (0.9126; 95% CI, 0.8810-0.9412) than the classical centralized approach (0.9045; 95% CI, 0.8701-0.9331) on an external test dataset (pairwise Wilcoxon signed-rank, P < .001). Notably, the federated approach performed significantly worse than the ensemble approach on both the holdout (0.8867; 95% CI, 0.8103-0.9481) and external test dataset (0.9227; 95% CI, 0.8941-0.9479).
Conclusions and Relevance: The findings of this diagnostic study suggest that federated learning is a viable approach for the binary classification of invasive melanomas and nevi on a clinically representative distributed dataset. Federated learning can improve privacy protection in AI-based melanoma diagnostics while simultaneously promoting collaboration across institutions and countries. Moreover, it may have the potential to be extended to other image classification tasks in digital cancer histopathology and beyond.
| Errataetall: |
CommentOn: JAMA Dermatol. 2024 Mar 1;160(3):269-270. - PMID 38324308 |
|---|---|
| Medienart: |
E-Artikel |
| Erscheinungsjahr: |
2024 |
|---|---|
| Erschienen: |
2024 |
| Enthalten in: |
Zur Gesamtaufnahme - volume:160 |
|---|---|
| Enthalten in: |
JAMA dermatology - 160(2024), 3 vom: 01. März, Seite 303-311 |
| Sprache: |
Englisch |
|---|
| Links: |
|---|
| Themen: |
|---|
| Anmerkungen: |
Date Completed 21.03.2024 Date Revised 21.03.2024 published: Print CommentOn: JAMA Dermatol. 2024 Mar 1;160(3):269-270. - PMID 38324308 Citation Status MEDLINE |
|---|
| doi: |
10.1001/jamadermatol.2023.5550 |
|---|
| funding: |
|
|---|---|
| Förderinstitution / Projekttitel: |
|
| PPN (Katalog-ID): |
NLM368134962 |
|---|
| LEADER | 01000caa a22002652 4500 | ||
|---|---|---|---|
| 001 | NLM368134962 | ||
| 003 | DE-627 | ||
| 005 | 20240322000023.0 | ||
| 007 | cr uuu---uuuuu | ||
| 008 | 240207s2024 xx |||||o 00| ||eng c | ||
| 024 | 7 | |a 10.1001/jamadermatol.2023.5550 |2 doi | |
| 028 | 5 | 2 | |a pubmed24n1338.xml |
| 035 | |a (DE-627)NLM368134962 | ||
| 035 | |a (NLM)38324293 | ||
| 040 | |a DE-627 |b ger |c DE-627 |e rakwb | ||
| 041 | |a eng | ||
| 100 | 1 | |a Haggenmüller, Sarah |e verfasserin |4 aut | |
| 245 | 1 | 0 | |a Federated Learning for Decentralized Artificial Intelligence in Melanoma Diagnostics |
| 264 | 1 | |c 2024 | |
| 336 | |a Text |b txt |2 rdacontent | ||
| 337 | |a ƒaComputermedien |b c |2 rdamedia | ||
| 338 | |a ƒa Online-Ressource |b cr |2 rdacarrier | ||
| 500 | |a Date Completed 21.03.2024 | ||
| 500 | |a Date Revised 21.03.2024 | ||
| 500 | |a published: Print | ||
| 500 | |a CommentOn: JAMA Dermatol. 2024 Mar 1;160(3):269-270. - PMID 38324308 | ||
| 500 | |a Citation Status MEDLINE | ||
| 520 | |a Importance: The development of artificial intelligence (AI)-based melanoma classifiers typically calls for large, centralized datasets, requiring hospitals to give away their patient data, which raises serious privacy concerns. To address this concern, decentralized federated learning has been proposed, where classifier development is distributed across hospitals | ||
| 520 | |a Objective: To investigate whether a more privacy-preserving federated learning approach can achieve comparable diagnostic performance to a classical centralized (ie, single-model) and ensemble learning approach for AI-based melanoma diagnostics | ||
| 520 | |a Design, Setting, and Participants: This multicentric, single-arm diagnostic study developed a federated model for melanoma-nevus classification using histopathological whole-slide images prospectively acquired at 6 German university hospitals between April 2021 and February 2023 and benchmarked it using both a holdout and an external test dataset. Data analysis was performed from February to April 2023 | ||
| 520 | |a Exposures: All whole-slide images were retrospectively analyzed by an AI-based classifier without influencing routine clinical care | ||
| 520 | |a Main Outcomes and Measures: The area under the receiver operating characteristic curve (AUROC) served as the primary end point for evaluating the diagnostic performance. Secondary end points included balanced accuracy, sensitivity, and specificity | ||
| 520 | |a Results: The study included 1025 whole-slide images of clinically melanoma-suspicious skin lesions from 923 patients, consisting of 388 histopathologically confirmed invasive melanomas and 637 nevi. The median (range) age at diagnosis was 58 (18-95) years for the training set, 57 (18-93) years for the holdout test dataset, and 61 (18-95) years for the external test dataset; the median (range) Breslow thickness was 0.70 (0.10-34.00) mm, 0.70 (0.20-14.40) mm, and 0.80 (0.30-20.00) mm, respectively. The federated approach (0.8579; 95% CI, 0.7693-0.9299) performed significantly worse than the classical centralized approach (0.9024; 95% CI, 0.8379-0.9565) in terms of AUROC on a holdout test dataset (pairwise Wilcoxon signed-rank, P < .001) but performed significantly better (0.9126; 95% CI, 0.8810-0.9412) than the classical centralized approach (0.9045; 95% CI, 0.8701-0.9331) on an external test dataset (pairwise Wilcoxon signed-rank, P < .001). Notably, the federated approach performed significantly worse than the ensemble approach on both the holdout (0.8867; 95% CI, 0.8103-0.9481) and external test dataset (0.9227; 95% CI, 0.8941-0.9479) | ||
| 520 | |a Conclusions and Relevance: The findings of this diagnostic study suggest that federated learning is a viable approach for the binary classification of invasive melanomas and nevi on a clinically representative distributed dataset. Federated learning can improve privacy protection in AI-based melanoma diagnostics while simultaneously promoting collaboration across institutions and countries. Moreover, it may have the potential to be extended to other image classification tasks in digital cancer histopathology and beyond | ||
| 650 | 4 | |a Journal Article | |
| 650 | 4 | |a Comment | |
| 700 | 1 | |a Schmitt, Max |e verfasserin |4 aut | |
| 700 | 1 | |a Krieghoff-Henning, Eva |e verfasserin |4 aut | |
| 700 | 1 | |a Hekler, Achim |e verfasserin |4 aut | |
| 700 | 1 | |a Maron, Roman C |e verfasserin |4 aut | |
| 700 | 1 | |a Wies, Christoph |e verfasserin |4 aut | |
| 700 | 1 | |a Utikal, Jochen S |e verfasserin |4 aut | |
| 700 | 1 | |a Meier, Friedegund |e verfasserin |4 aut | |
| 700 | 1 | |a Hobelsberger, Sarah |e verfasserin |4 aut | |
| 700 | 1 | |a Gellrich, Frank F |e verfasserin |4 aut | |
| 700 | 1 | |a Sergon, Mildred |e verfasserin |4 aut | |
| 700 | 1 | |a Hauschild, Axel |e verfasserin |4 aut | |
| 700 | 1 | |a French, Lars E |e verfasserin |4 aut | |
| 700 | 1 | |a Heinzerling, Lucie |e verfasserin |4 aut | |
| 700 | 1 | |a Schlager, Justin G |e verfasserin |4 aut | |
| 700 | 1 | |a Ghoreschi, Kamran |e verfasserin |4 aut | |
| 700 | 1 | |a Schlaak, Max |e verfasserin |4 aut | |
| 700 | 1 | |a Hilke, Franz J |e verfasserin |4 aut | |
| 700 | 1 | |a Poch, Gabriela |e verfasserin |4 aut | |
| 700 | 1 | |a Korsing, Sören |e verfasserin |4 aut | |
| 700 | 1 | |a Berking, Carola |e verfasserin |4 aut | |
| 700 | 1 | |a Heppt, Markus V |e verfasserin |4 aut | |
| 700 | 1 | |a Erdmann, Michael |e verfasserin |4 aut | |
| 700 | 1 | |a Haferkamp, Sebastian |e verfasserin |4 aut | |
| 700 | 1 | |a Drexler, Konstantin |e verfasserin |4 aut | |
| 700 | 1 | |a Schadendorf, Dirk |e verfasserin |4 aut | |
| 700 | 1 | |a Sondermann, Wiebke |e verfasserin |4 aut | |
| 700 | 1 | |a Goebeler, Matthias |e verfasserin |4 aut | |
| 700 | 1 | |a Schilling, Bastian |e verfasserin |4 aut | |
| 700 | 1 | |a Kather, Jakob N |e verfasserin |4 aut | |
| 700 | 1 | |a Fröhling, Stefan |e verfasserin |4 aut | |
| 700 | 1 | |a Brinker, Titus J |e verfasserin |4 aut | |
| 773 | 0 | 8 | |i Enthalten in |t JAMA dermatology |d 2013 |g 160(2024), 3 vom: 01. März, Seite 303-311 |w (DE-627)NLM221849696 |x 2168-6084 |7 nnns |
| 773 | 1 | 8 | |g volume:160 |g year:2024 |g number:3 |g day:01 |g month:03 |g pages:303-311 |
| 856 | 4 | 0 | |u http://dx.doi.org/10.1001/jamadermatol.2023.5550 |3 Volltext |
| 912 | |a GBV_USEFLAG_A | ||
| 912 | |a GBV_NLM | ||
| 951 | |a AR | ||
| 952 | |d 160 |j 2024 |e 3 |b 01 |c 03 |h 303-311 | ||