Multimodal, multitask, multiattention (M3) deep learning detection of reticular pseudodrusen : Toward automated and accessible classification of age-related macular degeneration
Published by Oxford University Press on behalf of the American Medical Informatics Association 2021. This work is written by US Government employees and is in the public domain in the US..
OBJECTIVE: Reticular pseudodrusen (RPD), a key feature of age-related macular degeneration (AMD), are poorly detected by human experts on standard color fundus photography (CFP) and typically require advanced imaging modalities such as fundus autofluorescence (FAF). The objective was to develop and evaluate the performance of a novel multimodal, multitask, multiattention (M3) deep learning framework on RPD detection.
MATERIALS AND METHODS: A deep learning framework (M3) was developed to detect RPD presence accurately using CFP alone, FAF alone, or both, employing >8000 CFP-FAF image pairs obtained prospectively (Age-Related Eye Disease Study 2). The M3 framework includes multimodal (detection from single or multiple image modalities), multitask (training different tasks simultaneously to improve generalizability), and multiattention (improving ensembled feature representation) operation. Performance on RPD detection was compared with state-of-the-art deep learning models and 13 ophthalmologists; performance on detection of 2 other AMD features (geographic atrophy and pigmentary abnormalities) was also evaluated.
RESULTS: For RPD detection, M3 achieved an area under the receiver-operating characteristic curve (AUROC) of 0.832, 0.931, and 0.933 for CFP alone, FAF alone, and both, respectively. M3 performance on CFP was very substantially superior to human retinal specialists (median F1 score = 0.644 vs 0.350). External validation (the Rotterdam Study) demonstrated high accuracy on CFP alone (AUROC, 0.965). The M3 framework also accurately detected geographic atrophy and pigmentary abnormalities (AUROC, 0.909 and 0.912, respectively), demonstrating its generalizability.
CONCLUSIONS: This study demonstrates the successful development, robust evaluation, and external validation of a novel deep learning framework that enables accessible, accurate, and automated AMD diagnosis and prognosis.
| Medienart: |
E-Artikel |
|---|
| Erscheinungsjahr: |
2021 |
|---|---|
| Erschienen: |
2021 |
| Enthalten in: |
Zur Gesamtaufnahme - volume:28 |
|---|---|
| Enthalten in: |
Journal of the American Medical Informatics Association : JAMIA - 28(2021), 6 vom: 12. Juni, Seite 1135-1148 |
| Sprache: |
Englisch |
|---|
| Beteiligte Personen: |
Chen, Qingyu [VerfasserIn] |
|---|
| Links: |
|---|
| Anmerkungen: |
Date Completed 18.08.2021 Date Revised 02.04.2024 published: Print Citation Status MEDLINE |
|---|
| doi: |
10.1093/jamia/ocaa302 |
|---|
| funding: |
|
|---|---|
| Förderinstitution / Projekttitel: |
|
| PPN (Katalog-ID): |
NLM323483011 |
|---|
| LEADER | 01000caa a22002652 4500 | ||
|---|---|---|---|
| 001 | NLM323483011 | ||
| 003 | DE-627 | ||
| 005 | 20240402232618.0 | ||
| 007 | cr uuu---uuuuu | ||
| 008 | 231225s2021 xx |||||o 00| ||eng c | ||
| 024 | 7 | |a 10.1093/jamia/ocaa302 |2 doi | |
| 028 | 5 | 2 | |a pubmed24n1360.xml |
| 035 | |a (DE-627)NLM323483011 | ||
| 035 | |a (NLM)33792724 | ||
| 040 | |a DE-627 |b ger |c DE-627 |e rakwb | ||
| 041 | |a eng | ||
| 100 | 1 | |a Chen, Qingyu |e verfasserin |4 aut | |
| 245 | 1 | 0 | |a Multimodal, multitask, multiattention (M3) deep learning detection of reticular pseudodrusen |b Toward automated and accessible classification of age-related macular degeneration |
| 264 | 1 | |c 2021 | |
| 336 | |a Text |b txt |2 rdacontent | ||
| 337 | |a ƒaComputermedien |b c |2 rdamedia | ||
| 338 | |a ƒa Online-Ressource |b cr |2 rdacarrier | ||
| 500 | |a Date Completed 18.08.2021 | ||
| 500 | |a Date Revised 02.04.2024 | ||
| 500 | |a published: Print | ||
| 500 | |a Citation Status MEDLINE | ||
| 520 | |a Published by Oxford University Press on behalf of the American Medical Informatics Association 2021. This work is written by US Government employees and is in the public domain in the US. | ||
| 520 | |a OBJECTIVE: Reticular pseudodrusen (RPD), a key feature of age-related macular degeneration (AMD), are poorly detected by human experts on standard color fundus photography (CFP) and typically require advanced imaging modalities such as fundus autofluorescence (FAF). The objective was to develop and evaluate the performance of a novel multimodal, multitask, multiattention (M3) deep learning framework on RPD detection | ||
| 520 | |a MATERIALS AND METHODS: A deep learning framework (M3) was developed to detect RPD presence accurately using CFP alone, FAF alone, or both, employing >8000 CFP-FAF image pairs obtained prospectively (Age-Related Eye Disease Study 2). The M3 framework includes multimodal (detection from single or multiple image modalities), multitask (training different tasks simultaneously to improve generalizability), and multiattention (improving ensembled feature representation) operation. Performance on RPD detection was compared with state-of-the-art deep learning models and 13 ophthalmologists; performance on detection of 2 other AMD features (geographic atrophy and pigmentary abnormalities) was also evaluated | ||
| 520 | |a RESULTS: For RPD detection, M3 achieved an area under the receiver-operating characteristic curve (AUROC) of 0.832, 0.931, and 0.933 for CFP alone, FAF alone, and both, respectively. M3 performance on CFP was very substantially superior to human retinal specialists (median F1 score = 0.644 vs 0.350). External validation (the Rotterdam Study) demonstrated high accuracy on CFP alone (AUROC, 0.965). The M3 framework also accurately detected geographic atrophy and pigmentary abnormalities (AUROC, 0.909 and 0.912, respectively), demonstrating its generalizability | ||
| 520 | |a CONCLUSIONS: This study demonstrates the successful development, robust evaluation, and external validation of a novel deep learning framework that enables accessible, accurate, and automated AMD diagnosis and prognosis | ||
| 650 | 4 | |a Journal Article | |
| 650 | 4 | |a Research Support, N.I.H., Extramural | |
| 650 | 4 | |a Research Support, N.I.H., Intramural | |
| 650 | 4 | |a Research Support, Non-U.S. Gov't | |
| 650 | 4 | |a Research Support, U.S. Gov't, Non-P.H.S. | |
| 650 | 4 | |a Age-Related Eye Disease Study 2 | |
| 650 | 4 | |a age-related macular degeneration | |
| 650 | 4 | |a deep learning | |
| 650 | 4 | |a multiattention deep learning | |
| 650 | 4 | |a multimodal deep learning | |
| 650 | 4 | |a multitask training | |
| 650 | 4 | |a reticular pseudodrusen | |
| 650 | 4 | |a subretinal drusenoid deposits | |
| 700 | 1 | |a Keenan, Tiarnan D L |e verfasserin |4 aut | |
| 700 | 1 | |a Allot, Alexis |e verfasserin |4 aut | |
| 700 | 1 | |a Peng, Yifan |e verfasserin |4 aut | |
| 700 | 1 | |a Agrón, Elvira |e verfasserin |4 aut | |
| 700 | 1 | |a Domalpally, Amitha |e verfasserin |4 aut | |
| 700 | 1 | |a Klaver, Caroline C W |e verfasserin |4 aut | |
| 700 | 1 | |a Luttikhuizen, Daniel T |e verfasserin |4 aut | |
| 700 | 1 | |a Colyer, Marcus H |e verfasserin |4 aut | |
| 700 | 1 | |a Cukras, Catherine A |e verfasserin |4 aut | |
| 700 | 1 | |a Wiley, Henry E |e verfasserin |4 aut | |
| 700 | 1 | |a Teresa Magone, M |e verfasserin |4 aut | |
| 700 | 1 | |a Cousineau-Krieger, Chantal |e verfasserin |4 aut | |
| 700 | 1 | |a Wong, Wai T |e verfasserin |4 aut | |
| 700 | 1 | |a Zhu, Yingying |e verfasserin |4 aut | |
| 700 | 1 | |a Chew, Emily Y |e verfasserin |4 aut | |
| 700 | 1 | |a Lu, Zhiyong |e verfasserin |4 aut | |
| 700 | 0 | |a AREDS2 Deep Learning Research Group |e verfasserin |4 aut | |
| 773 | 0 | 8 | |i Enthalten in |t Journal of the American Medical Informatics Association : JAMIA |d 1997 |g 28(2021), 6 vom: 12. Juni, Seite 1135-1148 |w (DE-627)NLM074735535 |x 1527-974X |7 nnns |
| 773 | 1 | 8 | |g volume:28 |g year:2021 |g number:6 |g day:12 |g month:06 |g pages:1135-1148 |
| 856 | 4 | 0 | |u http://dx.doi.org/10.1093/jamia/ocaa302 |3 Volltext |
| 912 | |a GBV_USEFLAG_A | ||
| 912 | |a GBV_NLM | ||
| 951 | |a AR | ||
| 952 | |d 28 |j 2021 |e 6 |b 12 |c 06 |h 1135-1148 | ||