Deep learning from multiple experts improves identification of amyloid neuropathologies
© 2022. The Author(s)..
Pathologists can label pathologies differently, making it challenging to yield consistent assessments in the absence of one ground truth. To address this problem, we present a deep learning (DL) approach that draws on a cohort of experts, weighs each contribution, and is robust to noisy labels. We collected 100,495 annotations on 20,099 candidate amyloid beta neuropathologies (cerebral amyloid angiopathy (CAA), and cored and diffuse plaques) from three institutions, independently annotated by five experts. DL methods trained on a consensus-of-two strategy yielded 12.6-26% improvements by area under the precision recall curve (AUPRC) when compared to those that learned individualized annotations. This strategy surpassed individual-expert models, even when unfairly assessed on benchmarks favoring them. Moreover, ensembling over individual models was robust to hidden random annotators. In blind prospective tests of 52,555 subsequent expert-annotated images, the models labeled pathologies like their human counterparts (consensus model AUPRC = 0.74 cored; 0.69 CAA). This study demonstrates a means to combine multiple ground truths into a common-ground DL model that yields consistent diagnoses informed by multiple and potentially variable expert opinions.
| Medienart: |
E-Artikel |
|---|
| Erscheinungsjahr: |
2022 |
|---|---|
| Erschienen: |
2022 |
| Enthalten in: |
Zur Gesamtaufnahme - volume:10 |
|---|---|
| Enthalten in: |
Acta neuropathologica communications - 10(2022), 1 vom: 28. Apr., Seite 66 |
| Sprache: |
Englisch |
|---|
| Beteiligte Personen: |
Wong, Daniel R [VerfasserIn] |
|---|
| Links: |
|---|
| Themen: |
Algorithms |
|---|
| Anmerkungen: |
Date Completed 02.05.2022 Date Revised 28.04.2023 published: Electronic Citation Status MEDLINE |
|---|
| doi: |
10.1186/s40478-022-01365-0 |
|---|
| funding: |
|
|---|---|
| Förderinstitution / Projekttitel: |
|
| PPN (Katalog-ID): |
NLM34013450X |
|---|
| LEADER | 01000naa a22002652 4500 | ||
|---|---|---|---|
| 001 | NLM34013450X | ||
| 003 | DE-627 | ||
| 005 | 20231226004445.0 | ||
| 007 | cr uuu---uuuuu | ||
| 008 | 231226s2022 xx |||||o 00| ||eng c | ||
| 024 | 7 | |a 10.1186/s40478-022-01365-0 |2 doi | |
| 028 | 5 | 2 | |a pubmed24n1133.xml |
| 035 | |a (DE-627)NLM34013450X | ||
| 035 | |a (NLM)35484610 | ||
| 040 | |a DE-627 |b ger |c DE-627 |e rakwb | ||
| 041 | |a eng | ||
| 100 | 1 | |a Wong, Daniel R |e verfasserin |4 aut | |
| 245 | 1 | 0 | |a Deep learning from multiple experts improves identification of amyloid neuropathologies |
| 264 | 1 | |c 2022 | |
| 336 | |a Text |b txt |2 rdacontent | ||
| 337 | |a ƒaComputermedien |b c |2 rdamedia | ||
| 338 | |a ƒa Online-Ressource |b cr |2 rdacarrier | ||
| 500 | |a Date Completed 02.05.2022 | ||
| 500 | |a Date Revised 28.04.2023 | ||
| 500 | |a published: Electronic | ||
| 500 | |a Citation Status MEDLINE | ||
| 520 | |a © 2022. The Author(s). | ||
| 520 | |a Pathologists can label pathologies differently, making it challenging to yield consistent assessments in the absence of one ground truth. To address this problem, we present a deep learning (DL) approach that draws on a cohort of experts, weighs each contribution, and is robust to noisy labels. We collected 100,495 annotations on 20,099 candidate amyloid beta neuropathologies (cerebral amyloid angiopathy (CAA), and cored and diffuse plaques) from three institutions, independently annotated by five experts. DL methods trained on a consensus-of-two strategy yielded 12.6-26% improvements by area under the precision recall curve (AUPRC) when compared to those that learned individualized annotations. This strategy surpassed individual-expert models, even when unfairly assessed on benchmarks favoring them. Moreover, ensembling over individual models was robust to hidden random annotators. In blind prospective tests of 52,555 subsequent expert-annotated images, the models labeled pathologies like their human counterparts (consensus model AUPRC = 0.74 cored; 0.69 CAA). This study demonstrates a means to combine multiple ground truths into a common-ground DL model that yields consistent diagnoses informed by multiple and potentially variable expert opinions | ||
| 650 | 4 | |a Journal Article | |
| 650 | 4 | |a Research Support, Non-U.S. Gov't | |
| 650 | 4 | |a Research Support, N.I.H., Extramural | |
| 650 | 4 | |a Algorithms | |
| 650 | 4 | |a Amyloid beta | |
| 650 | 4 | |a Consensus | |
| 650 | 4 | |a Deep learning | |
| 650 | 4 | |a Expert annotators | |
| 650 | 4 | |a Histopathology | |
| 650 | 7 | |a Amyloid beta-Peptides |2 NLM | |
| 700 | 1 | |a Tang, Ziqi |e verfasserin |4 aut | |
| 700 | 1 | |a Mew, Nicholas C |e verfasserin |4 aut | |
| 700 | 1 | |a Das, Sakshi |e verfasserin |4 aut | |
| 700 | 1 | |a Athey, Justin |e verfasserin |4 aut | |
| 700 | 1 | |a McAleese, Kirsty E |e verfasserin |4 aut | |
| 700 | 1 | |a Kofler, Julia K |e verfasserin |4 aut | |
| 700 | 1 | |a Flanagan, Margaret E |e verfasserin |4 aut | |
| 700 | 1 | |a Borys, Ewa |e verfasserin |4 aut | |
| 700 | 1 | |a White, Charles L |c 3rd |e verfasserin |4 aut | |
| 700 | 1 | |a Butte, Atul J |e verfasserin |4 aut | |
| 700 | 1 | |a Dugger, Brittany N |e verfasserin |4 aut | |
| 700 | 1 | |a Keiser, Michael J |e verfasserin |4 aut | |
| 773 | 0 | 8 | |i Enthalten in |t Acta neuropathologica communications |d 2013 |g 10(2022), 1 vom: 28. Apr., Seite 66 |w (DE-627)NLM230788246 |x 2051-5960 |7 nnns |
| 773 | 1 | 8 | |g volume:10 |g year:2022 |g number:1 |g day:28 |g month:04 |g pages:66 |
| 856 | 4 | 0 | |u http://dx.doi.org/10.1186/s40478-022-01365-0 |3 Volltext |
| 912 | |a GBV_USEFLAG_A | ||
| 912 | |a GBV_NLM | ||
| 951 | |a AR | ||
| 952 | |d 10 |j 2022 |e 1 |b 28 |c 04 |h 66 | ||