TGMIL : A hybrid multi-instance learning model based on the Transformer and the Graph Attention Network for whole-slide images classification of renal cell carcinoma
Copyright © 2023. Published by Elsevier B.V..
BACKGROUND AND OBJECTIVES: The pathological diagnosis of renal cell carcinoma is crucial for treatment. Currently, the multi-instance learning method is commonly used for whole-slide image classification of renal cell carcinoma, which is mainly based on the assumption of independent identical distribution. But this is inconsistent with the need to consider the correlation between different instances in the diagnosis process. Furthermore, the problem of high resource consumption of pathology images is still urgent to be solved. Therefore, we propose a new multi-instance learning method to solve this problem.
METHODS: In this study, we proposed a hybrid multi-instance learning model based on the Transformer and the Graph Attention Network, called TGMIL, to achieve whole-slide image of renal cell carcinoma classification without pixel-level annotation or region of interest extraction. Our approach is divided into three steps. First, we designed a feature pyramid with the multiple low magnifications of whole-slide image named MMFP. It makes the model incorporates richer information, and reduces memory consumption as well as training time compared to the highest magnification. Second, TGMIL amalgamates the Transformer and the Graph Attention's capabilities, adeptly addressing the loss of instance contextual and spatial. Within the Graph Attention network stream, an easy and efficient approach employing max pooling and mean pooling yields the graph adjacency matrix, devoid of extra memory consumption. Finally, the outputs of two streams of TGMIL are aggregated to achieve the classification of renal cell carcinoma.
RESULTS: On the TCGA-RCC validation set, a public dataset for renal cell carcinoma, the area under a receiver operating characteristic (ROC) curve (AUC) and accuracy of TGMIL were 0.98±0.0015,0.9191±0.0062, respectively. It showcased remarkable proficiency on the private validation set of renal cell carcinoma pathology images, attaining AUC of 0.9386±0.0162 and ACC of 0.9197±0.0124. Furthermore, on the public breast cancer whole-slide image test dataset, CAMELYON 16, our model showed good classification performance with an accuracy of 0.8792.
CONCLUSIONS: TGMIL models the diagnostic process of pathologists and shows good classification performance on multiple datasets. Concurrently, the MMFP module efficiently diminishes resource requirements, offering a novel angle for exploring computational pathology images.
| Medienart: |
E-Artikel |
|---|
| Erscheinungsjahr: |
2023 |
|---|---|
| Erschienen: |
2023 |
| Enthalten in: |
Zur Gesamtaufnahme - volume:242 |
|---|---|
| Enthalten in: |
Computer methods and programs in biomedicine - 242(2023) vom: 01. Dez., Seite 107789 |
| Sprache: |
Englisch |
|---|
| Beteiligte Personen: |
Sun, Xinhuan [VerfasserIn] |
|---|
| Links: |
|---|
| Themen: |
Graph attention network |
|---|
| Anmerkungen: |
Date Completed 14.11.2023 Date Revised 14.11.2023 published: Print-Electronic Citation Status MEDLINE |
|---|
| doi: |
10.1016/j.cmpb.2023.107789 |
|---|
| funding: |
|
|---|---|
| Förderinstitution / Projekttitel: |
|
| PPN (Katalog-ID): |
NLM362191484 |
|---|
| LEADER | 01000naa a22002652 4500 | ||
|---|---|---|---|
| 001 | NLM362191484 | ||
| 003 | DE-627 | ||
| 005 | 20231226090728.0 | ||
| 007 | cr uuu---uuuuu | ||
| 008 | 231226s2023 xx |||||o 00| ||eng c | ||
| 024 | 7 | |a 10.1016/j.cmpb.2023.107789 |2 doi | |
| 028 | 5 | 2 | |a pubmed24n1207.xml |
| 035 | |a (DE-627)NLM362191484 | ||
| 035 | |a (NLM)37722310 | ||
| 035 | |a (PII)S0169-2607(23)00455-8 | ||
| 040 | |a DE-627 |b ger |c DE-627 |e rakwb | ||
| 041 | |a eng | ||
| 100 | 1 | |a Sun, Xinhuan |e verfasserin |4 aut | |
| 245 | 1 | 0 | |a TGMIL |b A hybrid multi-instance learning model based on the Transformer and the Graph Attention Network for whole-slide images classification of renal cell carcinoma |
| 264 | 1 | |c 2023 | |
| 336 | |a Text |b txt |2 rdacontent | ||
| 337 | |a ƒaComputermedien |b c |2 rdamedia | ||
| 338 | |a ƒa Online-Ressource |b cr |2 rdacarrier | ||
| 500 | |a Date Completed 14.11.2023 | ||
| 500 | |a Date Revised 14.11.2023 | ||
| 500 | |a published: Print-Electronic | ||
| 500 | |a Citation Status MEDLINE | ||
| 520 | |a Copyright © 2023. Published by Elsevier B.V. | ||
| 520 | |a BACKGROUND AND OBJECTIVES: The pathological diagnosis of renal cell carcinoma is crucial for treatment. Currently, the multi-instance learning method is commonly used for whole-slide image classification of renal cell carcinoma, which is mainly based on the assumption of independent identical distribution. But this is inconsistent with the need to consider the correlation between different instances in the diagnosis process. Furthermore, the problem of high resource consumption of pathology images is still urgent to be solved. Therefore, we propose a new multi-instance learning method to solve this problem | ||
| 520 | |a METHODS: In this study, we proposed a hybrid multi-instance learning model based on the Transformer and the Graph Attention Network, called TGMIL, to achieve whole-slide image of renal cell carcinoma classification without pixel-level annotation or region of interest extraction. Our approach is divided into three steps. First, we designed a feature pyramid with the multiple low magnifications of whole-slide image named MMFP. It makes the model incorporates richer information, and reduces memory consumption as well as training time compared to the highest magnification. Second, TGMIL amalgamates the Transformer and the Graph Attention's capabilities, adeptly addressing the loss of instance contextual and spatial. Within the Graph Attention network stream, an easy and efficient approach employing max pooling and mean pooling yields the graph adjacency matrix, devoid of extra memory consumption. Finally, the outputs of two streams of TGMIL are aggregated to achieve the classification of renal cell carcinoma | ||
| 520 | |a RESULTS: On the TCGA-RCC validation set, a public dataset for renal cell carcinoma, the area under a receiver operating characteristic (ROC) curve (AUC) and accuracy of TGMIL were 0.98±0.0015,0.9191±0.0062, respectively. It showcased remarkable proficiency on the private validation set of renal cell carcinoma pathology images, attaining AUC of 0.9386±0.0162 and ACC of 0.9197±0.0124. Furthermore, on the public breast cancer whole-slide image test dataset, CAMELYON 16, our model showed good classification performance with an accuracy of 0.8792 | ||
| 520 | |a CONCLUSIONS: TGMIL models the diagnostic process of pathologists and shows good classification performance on multiple datasets. Concurrently, the MMFP module efficiently diminishes resource requirements, offering a novel angle for exploring computational pathology images | ||
| 650 | 4 | |a Journal Article | |
| 650 | 4 | |a Graph attention network | |
| 650 | 4 | |a Multi-instance learning | |
| 650 | 4 | |a Renal cell carcinoma | |
| 650 | 4 | |a Transformer | |
| 650 | 4 | |a Whole-slide image | |
| 700 | 1 | |a Li, Wuchao |e verfasserin |4 aut | |
| 700 | 1 | |a Fu, Bangkang |e verfasserin |4 aut | |
| 700 | 1 | |a Peng, Yunsong |e verfasserin |4 aut | |
| 700 | 1 | |a He, Junjie |e verfasserin |4 aut | |
| 700 | 1 | |a Wang, Lihui |e verfasserin |4 aut | |
| 700 | 1 | |a Yang, Tongyin |e verfasserin |4 aut | |
| 700 | 1 | |a Meng, Xue |e verfasserin |4 aut | |
| 700 | 1 | |a Li, Jin |e verfasserin |4 aut | |
| 700 | 1 | |a Wang, Jinjing |e verfasserin |4 aut | |
| 700 | 1 | |a Huang, Ping |e verfasserin |4 aut | |
| 700 | 1 | |a Wang, Rongpin |e verfasserin |4 aut | |
| 773 | 0 | 8 | |i Enthalten in |t Computer methods and programs in biomedicine |d 1993 |g 242(2023) vom: 01. Dez., Seite 107789 |w (DE-627)NLM012836133 |x 1872-7565 |7 nnns |
| 773 | 1 | 8 | |g volume:242 |g year:2023 |g day:01 |g month:12 |g pages:107789 |
| 856 | 4 | 0 | |u http://dx.doi.org/10.1016/j.cmpb.2023.107789 |3 Volltext |
| 912 | |a GBV_USEFLAG_A | ||
| 912 | |a GBV_NLM | ||
| 951 | |a AR | ||
| 952 | |d 242 |j 2023 |b 01 |c 12 |h 107789 | ||