Multimodal graph attention network for COVID-19 outcome prediction
© 2023. The Author(s)..
When dealing with a newly emerging disease such as COVID-19, the impact of patient- and disease-specific factors (e.g., body weight or known co-morbidities) on the immediate course of the disease is largely unknown. An accurate prediction of the most likely individual disease progression can improve the planning of limited resources and finding the optimal treatment for patients. In the case of COVID-19, the need for intensive care unit (ICU) admission of pneumonia patients can often only be determined on short notice by acute indicators such as vital signs (e.g., breathing rate, blood oxygen levels), whereas statistical analysis and decision support systems that integrate all of the available data could enable an earlier prognosis. To this end, we propose a holistic, multimodal graph-based approach combining imaging and non-imaging information. Specifically, we introduce a multimodal similarity metric to build a population graph that shows a clustering of patients. For each patient in the graph, we extract radiomic features from a segmentation network that also serves as a latent image feature encoder. Together with clinical patient data like vital signs, demographics, and lab results, these modalities are combined into a multimodal representation of each patient. This feature extraction is trained end-to-end with an image-based Graph Attention Network to process the population graph and predict the COVID-19 patient outcomes: admission to ICU, need for ventilation, and mortality. To combine multiple modalities, radiomic features are extracted from chest CTs using a segmentation neural network. Results on a dataset collected in Klinikum rechts der Isar in Munich, Germany and the publicly available iCTCF dataset show that our approach outperforms single modality and non-graph baselines. Moreover, our clustering and graph attention increases understanding of the patient relationships within the population graph and provides insight into the network's decision-making process.
| Medienart: |
E-Artikel |
|---|
| Erscheinungsjahr: |
2023 |
|---|---|
| Erschienen: |
2023 |
| Enthalten in: |
Zur Gesamtaufnahme - volume:13 |
|---|---|
| Enthalten in: |
Scientific reports - 13(2023), 1 vom: 09. Nov., Seite 19539 |
| Sprache: |
Englisch |
|---|
| Beteiligte Personen: |
Keicher, Matthias [VerfasserIn] |
|---|
| Links: |
|---|
| Themen: |
|---|
| Anmerkungen: |
Date Completed 13.11.2023 Date Revised 13.11.2023 published: Electronic Citation Status MEDLINE |
|---|
| doi: |
10.1038/s41598-023-46625-8 |
|---|
| funding: |
|
|---|---|
| Förderinstitution / Projekttitel: |
|
| PPN (Katalog-ID): |
NLM364373008 |
|---|
| LEADER | 01000naa a22002652 4500 | ||
|---|---|---|---|
| 001 | NLM364373008 | ||
| 003 | DE-627 | ||
| 005 | 20231226211142.0 | ||
| 007 | cr uuu---uuuuu | ||
| 008 | 231226s2023 xx |||||o 00| ||eng c | ||
| 024 | 7 | |a 10.1038/s41598-023-46625-8 |2 doi | |
| 028 | 5 | 2 | |a pubmed24n1214.xml |
| 035 | |a (DE-627)NLM364373008 | ||
| 035 | |a (NLM)37945590 | ||
| 040 | |a DE-627 |b ger |c DE-627 |e rakwb | ||
| 041 | |a eng | ||
| 100 | 1 | |a Keicher, Matthias |e verfasserin |4 aut | |
| 245 | 1 | 0 | |a Multimodal graph attention network for COVID-19 outcome prediction |
| 264 | 1 | |c 2023 | |
| 336 | |a Text |b txt |2 rdacontent | ||
| 337 | |a ƒaComputermedien |b c |2 rdamedia | ||
| 338 | |a ƒa Online-Ressource |b cr |2 rdacarrier | ||
| 500 | |a Date Completed 13.11.2023 | ||
| 500 | |a Date Revised 13.11.2023 | ||
| 500 | |a published: Electronic | ||
| 500 | |a Citation Status MEDLINE | ||
| 520 | |a © 2023. The Author(s). | ||
| 520 | |a When dealing with a newly emerging disease such as COVID-19, the impact of patient- and disease-specific factors (e.g., body weight or known co-morbidities) on the immediate course of the disease is largely unknown. An accurate prediction of the most likely individual disease progression can improve the planning of limited resources and finding the optimal treatment for patients. In the case of COVID-19, the need for intensive care unit (ICU) admission of pneumonia patients can often only be determined on short notice by acute indicators such as vital signs (e.g., breathing rate, blood oxygen levels), whereas statistical analysis and decision support systems that integrate all of the available data could enable an earlier prognosis. To this end, we propose a holistic, multimodal graph-based approach combining imaging and non-imaging information. Specifically, we introduce a multimodal similarity metric to build a population graph that shows a clustering of patients. For each patient in the graph, we extract radiomic features from a segmentation network that also serves as a latent image feature encoder. Together with clinical patient data like vital signs, demographics, and lab results, these modalities are combined into a multimodal representation of each patient. This feature extraction is trained end-to-end with an image-based Graph Attention Network to process the population graph and predict the COVID-19 patient outcomes: admission to ICU, need for ventilation, and mortality. To combine multiple modalities, radiomic features are extracted from chest CTs using a segmentation neural network. Results on a dataset collected in Klinikum rechts der Isar in Munich, Germany and the publicly available iCTCF dataset show that our approach outperforms single modality and non-graph baselines. Moreover, our clustering and graph attention increases understanding of the patient relationships within the population graph and provides insight into the network's decision-making process | ||
| 650 | 4 | |a Journal Article | |
| 650 | 4 | |a Research Support, Non-U.S. Gov't | |
| 700 | 1 | |a Burwinkel, Hendrik |e verfasserin |4 aut | |
| 700 | 1 | |a Bani-Harouni, David |e verfasserin |4 aut | |
| 700 | 1 | |a Paschali, Magdalini |e verfasserin |4 aut | |
| 700 | 1 | |a Czempiel, Tobias |e verfasserin |4 aut | |
| 700 | 1 | |a Burian, Egon |e verfasserin |4 aut | |
| 700 | 1 | |a Makowski, Marcus R |e verfasserin |4 aut | |
| 700 | 1 | |a Braren, Rickmer |e verfasserin |4 aut | |
| 700 | 1 | |a Navab, Nassir |e verfasserin |4 aut | |
| 700 | 1 | |a Wendler, Thomas |e verfasserin |4 aut | |
| 773 | 0 | 8 | |i Enthalten in |t Scientific reports |d 2011 |g 13(2023), 1 vom: 09. Nov., Seite 19539 |w (DE-627)NLM215703936 |x 2045-2322 |7 nnns |
| 773 | 1 | 8 | |g volume:13 |g year:2023 |g number:1 |g day:09 |g month:11 |g pages:19539 |
| 856 | 4 | 0 | |u http://dx.doi.org/10.1038/s41598-023-46625-8 |3 Volltext |
| 912 | |a GBV_USEFLAG_A | ||
| 912 | |a GBV_NLM | ||
| 951 | |a AR | ||
| 952 | |d 13 |j 2023 |e 1 |b 09 |c 11 |h 19539 | ||