GMRLNet : A Graph-Based Manifold Regularization Learning Framework for Placental Insufficiency Diagnosis on Incomplete Multimodal Ultrasound Data
Multimodal analysis of placental ultrasound (US) and microflow imaging (MFI) could greatly aid in the early diagnosis and interventional treatment of placental insufficiency (PI), ensuring a normal pregnancy. Existing multimodal analysis methods have weaknesses in multimodal feature representation and modal knowledge definitions and fail on incomplete datasets with unpaired multimodal samples. To address these challenges and efficiently leverage the incomplete multimodal dataset for accurate PI diagnosis, we propose a novel graph-based manifold regularization learning (MRL) framework named GMRLNet. It takes US and MFI images as input and exploits their modality-shared and modality-specific information for optimal multimodal feature representation. Specifically, a graph convolutional-based shared and specific transfer network (GSSTN) is designed to explore intra-modal feature associations, thus decoupling each modal input into interpretable shared and specific spaces. For unimodal knowledge definitions, graph-based manifold knowledge is introduced to describe the sample-level feature representation, local inter-sample relations, and global data distribution of each modality. Then, an MRL paradigm is designed for inter-modal manifold knowledge transfer to obtain effective cross-modal feature representations. Furthermore, MRL transfers the knowledge between both paired and unpaired data for robust learning on incomplete datasets. Experiments were conducted on two clinical datasets to validate the PI classification performance and generalization of GMRLNet. State-of-the-art comparisons show the higher accuracy of GMRLNet on incomplete datasets. Our method achieves 0.913 AUC and 0.904 balanced accuracy (bACC) for paired US and MFI images, as well as 0.906 AUC and 0.888 bACC for unimodal US images, illustrating its application potential in PI CAD systems.
| Medienart: |
E-Artikel |
|---|
| Erscheinungsjahr: |
2023 |
|---|---|
| Erschienen: |
2023 |
| Enthalten in: |
Zur Gesamtaufnahme - volume:42 |
|---|---|
| Enthalten in: |
IEEE transactions on medical imaging - 42(2023), 11 vom: 22. Nov., Seite 3205-3218 |
| Sprache: |
Englisch |
|---|
| Beteiligte Personen: |
Jiao, Jing [VerfasserIn] |
|---|
| Links: |
|---|
| Themen: |
|---|
| Anmerkungen: |
Date Completed 30.10.2023 Date Revised 30.10.2023 published: Print-Electronic Citation Status MEDLINE |
|---|
| doi: |
10.1109/TMI.2023.3278259 |
|---|
| funding: |
|
|---|---|
| Förderinstitution / Projekttitel: |
|
| PPN (Katalog-ID): |
NLM357187237 |
|---|
| LEADER | 01000naa a22002652 4500 | ||
|---|---|---|---|
| 001 | NLM357187237 | ||
| 003 | DE-627 | ||
| 005 | 20231226072039.0 | ||
| 007 | cr uuu---uuuuu | ||
| 008 | 231226s2023 xx |||||o 00| ||eng c | ||
| 024 | 7 | |a 10.1109/TMI.2023.3278259 |2 doi | |
| 028 | 5 | 2 | |a pubmed24n1190.xml |
| 035 | |a (DE-627)NLM357187237 | ||
| 035 | |a (NLM)37216245 | ||
| 040 | |a DE-627 |b ger |c DE-627 |e rakwb | ||
| 041 | |a eng | ||
| 100 | 1 | |a Jiao, Jing |e verfasserin |4 aut | |
| 245 | 1 | 0 | |a GMRLNet |b A Graph-Based Manifold Regularization Learning Framework for Placental Insufficiency Diagnosis on Incomplete Multimodal Ultrasound Data |
| 264 | 1 | |c 2023 | |
| 336 | |a Text |b txt |2 rdacontent | ||
| 337 | |a ƒaComputermedien |b c |2 rdamedia | ||
| 338 | |a ƒa Online-Ressource |b cr |2 rdacarrier | ||
| 500 | |a Date Completed 30.10.2023 | ||
| 500 | |a Date Revised 30.10.2023 | ||
| 500 | |a published: Print-Electronic | ||
| 500 | |a Citation Status MEDLINE | ||
| 520 | |a Multimodal analysis of placental ultrasound (US) and microflow imaging (MFI) could greatly aid in the early diagnosis and interventional treatment of placental insufficiency (PI), ensuring a normal pregnancy. Existing multimodal analysis methods have weaknesses in multimodal feature representation and modal knowledge definitions and fail on incomplete datasets with unpaired multimodal samples. To address these challenges and efficiently leverage the incomplete multimodal dataset for accurate PI diagnosis, we propose a novel graph-based manifold regularization learning (MRL) framework named GMRLNet. It takes US and MFI images as input and exploits their modality-shared and modality-specific information for optimal multimodal feature representation. Specifically, a graph convolutional-based shared and specific transfer network (GSSTN) is designed to explore intra-modal feature associations, thus decoupling each modal input into interpretable shared and specific spaces. For unimodal knowledge definitions, graph-based manifold knowledge is introduced to describe the sample-level feature representation, local inter-sample relations, and global data distribution of each modality. Then, an MRL paradigm is designed for inter-modal manifold knowledge transfer to obtain effective cross-modal feature representations. Furthermore, MRL transfers the knowledge between both paired and unpaired data for robust learning on incomplete datasets. Experiments were conducted on two clinical datasets to validate the PI classification performance and generalization of GMRLNet. State-of-the-art comparisons show the higher accuracy of GMRLNet on incomplete datasets. Our method achieves 0.913 AUC and 0.904 balanced accuracy (bACC) for paired US and MFI images, as well as 0.906 AUC and 0.888 bACC for unimodal US images, illustrating its application potential in PI CAD systems | ||
| 650 | 4 | |a Journal Article | |
| 650 | 4 | |a Research Support, Non-U.S. Gov't | |
| 700 | 1 | |a Sun, Hongshuang |e verfasserin |4 aut | |
| 700 | 1 | |a Huang, Yi |e verfasserin |4 aut | |
| 700 | 1 | |a Xia, Menghua |e verfasserin |4 aut | |
| 700 | 1 | |a Qiao, Mengyun |e verfasserin |4 aut | |
| 700 | 1 | |a Ren, Yunyun |e verfasserin |4 aut | |
| 700 | 1 | |a Wang, Yuanyuan |e verfasserin |4 aut | |
| 700 | 1 | |a Guo, Yi |e verfasserin |4 aut | |
| 773 | 0 | 8 | |i Enthalten in |t IEEE transactions on medical imaging |d 1982 |g 42(2023), 11 vom: 22. Nov., Seite 3205-3218 |w (DE-627)NLM082855269 |x 1558-254X |7 nnns |
| 773 | 1 | 8 | |g volume:42 |g year:2023 |g number:11 |g day:22 |g month:11 |g pages:3205-3218 |
| 856 | 4 | 0 | |u http://dx.doi.org/10.1109/TMI.2023.3278259 |3 Volltext |
| 912 | |a GBV_USEFLAG_A | ||
| 912 | |a GBV_NLM | ||
| 951 | |a AR | ||
| 952 | |d 42 |j 2023 |e 11 |b 22 |c 11 |h 3205-3218 | ||