Multi-Scale Tokens-Aware Transformer Network for Multi-Region and Multi-Sequence MR-to-CT Synthesis in a Single Model
The superiority of magnetic resonance (MR)-only radiotherapy treatment planning (RTP) has been well demonstrated, benefiting from the synthesis of computed tomography (CT) images which supplements electron density and eliminates the errors of multi-modal images registration. An increasing number of methods has been proposed for MR-to-CT synthesis. However, synthesizing CT images of different anatomical regions from MR images with different sequences using a single model is challenging due to the large differences between these regions and the limitations of convolutional neural networks in capturing global context information. In this paper, we propose a multi-scale tokens-aware Transformer network (MTT-Net) for multi-region and multi-sequence MR-to-CT synthesis in a single model. Specifically, we develop a multi-scale image tokens Transformer to capture multi-scale global spatial information between different anatomical structures in different regions. Besides, to address the limited attention areas of tokens in Transformer, we introduce a multi-shape window self-attention into Transformer to enlarge the receptive fields for learning the multi-directional spatial representations. Moreover, we adopt a domain classifier in generator to introduce the domain knowledge for distinguishing the MR images of different regions and sequences. The proposed MTT-Net is evaluated on a multi-center dataset and an unseen region, and remarkable performance was achieved with MAE of 69.33 ± 10.39 HU, SSIM of 0.778 ± 0.028, and PSNR of 29.04 ± 1.32 dB in head & neck region, and MAE of 62.80 ± 7.65 HU, SSIM of 0.617 ± 0.058 and PSNR of 25.94 ± 1.02 dB in abdomen region. The proposed MTT-Net outperforms state-of-the-art methods in both accuracy and visual quality.
| Medienart: |
E-Artikel |
|---|
| Erscheinungsjahr: |
2024 |
|---|---|
| Erschienen: |
2024 |
| Enthalten in: |
Zur Gesamtaufnahme - volume:43 |
|---|---|
| Enthalten in: |
IEEE transactions on medical imaging - 43(2024), 2 vom: 01. Feb., Seite 794-806 |
| Sprache: |
Englisch |
|---|
| Beteiligte Personen: |
Zhong, Liming [VerfasserIn] |
|---|
| Links: |
|---|
| Themen: |
|---|
| Anmerkungen: |
Date Completed 05.02.2024 Date Revised 05.02.2024 published: Print-Electronic Citation Status MEDLINE |
|---|
| doi: |
10.1109/TMI.2023.3321064 |
|---|
| funding: |
|
|---|---|
| Förderinstitution / Projekttitel: |
|
| PPN (Katalog-ID): |
NLM362784264 |
|---|
| LEADER | 01000caa a22002652 4500 | ||
|---|---|---|---|
| 001 | NLM362784264 | ||
| 003 | DE-627 | ||
| 005 | 20240205231901.0 | ||
| 007 | cr uuu---uuuuu | ||
| 008 | 231226s2024 xx |||||o 00| ||eng c | ||
| 024 | 7 | |a 10.1109/TMI.2023.3321064 |2 doi | |
| 028 | 5 | 2 | |a pubmed24n1281.xml |
| 035 | |a (DE-627)NLM362784264 | ||
| 035 | |a (NLM)37782590 | ||
| 040 | |a DE-627 |b ger |c DE-627 |e rakwb | ||
| 041 | |a eng | ||
| 100 | 1 | |a Zhong, Liming |e verfasserin |4 aut | |
| 245 | 1 | 0 | |a Multi-Scale Tokens-Aware Transformer Network for Multi-Region and Multi-Sequence MR-to-CT Synthesis in a Single Model |
| 264 | 1 | |c 2024 | |
| 336 | |a Text |b txt |2 rdacontent | ||
| 337 | |a ƒaComputermedien |b c |2 rdamedia | ||
| 338 | |a ƒa Online-Ressource |b cr |2 rdacarrier | ||
| 500 | |a Date Completed 05.02.2024 | ||
| 500 | |a Date Revised 05.02.2024 | ||
| 500 | |a published: Print-Electronic | ||
| 500 | |a Citation Status MEDLINE | ||
| 520 | |a The superiority of magnetic resonance (MR)-only radiotherapy treatment planning (RTP) has been well demonstrated, benefiting from the synthesis of computed tomography (CT) images which supplements electron density and eliminates the errors of multi-modal images registration. An increasing number of methods has been proposed for MR-to-CT synthesis. However, synthesizing CT images of different anatomical regions from MR images with different sequences using a single model is challenging due to the large differences between these regions and the limitations of convolutional neural networks in capturing global context information. In this paper, we propose a multi-scale tokens-aware Transformer network (MTT-Net) for multi-region and multi-sequence MR-to-CT synthesis in a single model. Specifically, we develop a multi-scale image tokens Transformer to capture multi-scale global spatial information between different anatomical structures in different regions. Besides, to address the limited attention areas of tokens in Transformer, we introduce a multi-shape window self-attention into Transformer to enlarge the receptive fields for learning the multi-directional spatial representations. Moreover, we adopt a domain classifier in generator to introduce the domain knowledge for distinguishing the MR images of different regions and sequences. The proposed MTT-Net is evaluated on a multi-center dataset and an unseen region, and remarkable performance was achieved with MAE of 69.33 ± 10.39 HU, SSIM of 0.778 ± 0.028, and PSNR of 29.04 ± 1.32 dB in head & neck region, and MAE of 62.80 ± 7.65 HU, SSIM of 0.617 ± 0.058 and PSNR of 25.94 ± 1.02 dB in abdomen region. The proposed MTT-Net outperforms state-of-the-art methods in both accuracy and visual quality | ||
| 650 | 4 | |a Journal Article | |
| 700 | 1 | |a Chen, Zeli |e verfasserin |4 aut | |
| 700 | 1 | |a Shu, Hai |e verfasserin |4 aut | |
| 700 | 1 | |a Zheng, Kaiyi |e verfasserin |4 aut | |
| 700 | 1 | |a Li, Yin |e verfasserin |4 aut | |
| 700 | 1 | |a Chen, Weicui |e verfasserin |4 aut | |
| 700 | 1 | |a Wu, Yuankui |e verfasserin |4 aut | |
| 700 | 1 | |a Ma, Jianhua |e verfasserin |4 aut | |
| 700 | 1 | |a Feng, Qianjin |e verfasserin |4 aut | |
| 700 | 1 | |a Yang, Wei |e verfasserin |4 aut | |
| 773 | 0 | 8 | |i Enthalten in |t IEEE transactions on medical imaging |d 1982 |g 43(2024), 2 vom: 01. Feb., Seite 794-806 |w (DE-627)NLM082855269 |x 1558-254X |7 nnns |
| 773 | 1 | 8 | |g volume:43 |g year:2024 |g number:2 |g day:01 |g month:02 |g pages:794-806 |
| 856 | 4 | 0 | |u http://dx.doi.org/10.1109/TMI.2023.3321064 |3 Volltext |
| 912 | |a GBV_USEFLAG_A | ||
| 912 | |a GBV_NLM | ||
| 951 | |a AR | ||
| 952 | |d 43 |j 2024 |e 2 |b 01 |c 02 |h 794-806 | ||