Hole-filling based on content loss indexed 3D partial convolution network for freehand ultrasound reconstruction
Copyright © 2021. Published by Elsevier B.V..
BACKGROUND AND OBJECTIVE: During the 3D reconstruction of ultrasound volume from 2D B-scan ultrasound images, holes are usually found in the reconstructed 3D volumes due to the fast scans. This condition will affect the positioning and judgment of the doctor to the lesion. Hence, in this study, we propose to fill the holes by using a novel content loss indexed 3D partial convolution network for 3D freehand ultrasound volume reconstruction. The network can synthesize novel ultrasound volume structures and reconstruct ultrasound volume with missing regions with variable sizes and at arbitrary locations.
METHODS: First, the 3D partial convolution is introduced into the convolutional layer, which is masked and renormalized to be conditioned on only valid voxels. Then, the mask in the next layer is automatically updated as a part of the forward pass. To better preserve texture and structure details of the reconstruction results, we couple the adversarial loss of the least squares generative adversarial network (LSGAN) with the innovative content loss, which consists of the context loss, the feature-matching loss and the total variation loss. Thereafter, we introduce a novel spectral-normalized LSGAN by adding spectral normalization (SN) to the generator and discriminator of the LSGAN. The proposed method is simple in formulation, and is stable in training.
RESULTS: Experiments on public and in-vivo ultrasound datasets and comparisons with popular algorithms demonstrate that the proposed approach can generate high-quality hole-filling results with preserved perceptual image details.
CONCLUSIONS: Considering the high quality of the hole-filling results, the proposed method can effectively fill the missing regions in the reconstructed 3D ultrasound volume from 2D ultrasound image sequences.
| Medienart: |
E-Artikel |
|---|
| Erscheinungsjahr: |
2021 |
|---|---|
| Erschienen: |
2021 |
| Enthalten in: |
Zur Gesamtaufnahme - volume:211 |
|---|---|
| Enthalten in: |
Computer methods and programs in biomedicine - 211(2021) vom: 25. Nov., Seite 106421 |
| Sprache: |
Englisch |
|---|
| Beteiligte Personen: |
Dong, Jiahui [VerfasserIn] |
|---|
| Links: |
|---|
| Themen: |
3D reconstruction |
|---|
| Anmerkungen: |
Date Completed 28.10.2021 Date Revised 28.10.2021 published: Print-Electronic Citation Status MEDLINE |
|---|
| doi: |
10.1016/j.cmpb.2021.106421 |
|---|
| funding: |
|
|---|---|
| Förderinstitution / Projekttitel: |
|
| PPN (Katalog-ID): |
NLM331247747 |
|---|
| LEADER | 01000naa a22002652 4500 | ||
|---|---|---|---|
| 001 | NLM331247747 | ||
| 003 | DE-627 | ||
| 005 | 20231225213039.0 | ||
| 007 | cr uuu---uuuuu | ||
| 008 | 231225s2021 xx |||||o 00| ||eng c | ||
| 024 | 7 | |a 10.1016/j.cmpb.2021.106421 |2 doi | |
| 028 | 5 | 2 | |a pubmed24n1104.xml |
| 035 | |a (DE-627)NLM331247747 | ||
| 035 | |a (NLM)34583228 | ||
| 035 | |a (PII)S0169-2607(21)00495-8 | ||
| 040 | |a DE-627 |b ger |c DE-627 |e rakwb | ||
| 041 | |a eng | ||
| 100 | 1 | |a Dong, Jiahui |e verfasserin |4 aut | |
| 245 | 1 | 0 | |a Hole-filling based on content loss indexed 3D partial convolution network for freehand ultrasound reconstruction |
| 264 | 1 | |c 2021 | |
| 336 | |a Text |b txt |2 rdacontent | ||
| 337 | |a ƒaComputermedien |b c |2 rdamedia | ||
| 338 | |a ƒa Online-Ressource |b cr |2 rdacarrier | ||
| 500 | |a Date Completed 28.10.2021 | ||
| 500 | |a Date Revised 28.10.2021 | ||
| 500 | |a published: Print-Electronic | ||
| 500 | |a Citation Status MEDLINE | ||
| 520 | |a Copyright © 2021. Published by Elsevier B.V. | ||
| 520 | |a BACKGROUND AND OBJECTIVE: During the 3D reconstruction of ultrasound volume from 2D B-scan ultrasound images, holes are usually found in the reconstructed 3D volumes due to the fast scans. This condition will affect the positioning and judgment of the doctor to the lesion. Hence, in this study, we propose to fill the holes by using a novel content loss indexed 3D partial convolution network for 3D freehand ultrasound volume reconstruction. The network can synthesize novel ultrasound volume structures and reconstruct ultrasound volume with missing regions with variable sizes and at arbitrary locations | ||
| 520 | |a METHODS: First, the 3D partial convolution is introduced into the convolutional layer, which is masked and renormalized to be conditioned on only valid voxels. Then, the mask in the next layer is automatically updated as a part of the forward pass. To better preserve texture and structure details of the reconstruction results, we couple the adversarial loss of the least squares generative adversarial network (LSGAN) with the innovative content loss, which consists of the context loss, the feature-matching loss and the total variation loss. Thereafter, we introduce a novel spectral-normalized LSGAN by adding spectral normalization (SN) to the generator and discriminator of the LSGAN. The proposed method is simple in formulation, and is stable in training | ||
| 520 | |a RESULTS: Experiments on public and in-vivo ultrasound datasets and comparisons with popular algorithms demonstrate that the proposed approach can generate high-quality hole-filling results with preserved perceptual image details | ||
| 520 | |a CONCLUSIONS: Considering the high quality of the hole-filling results, the proposed method can effectively fill the missing regions in the reconstructed 3D ultrasound volume from 2D ultrasound image sequences | ||
| 650 | 4 | |a Journal Article | |
| 650 | 4 | |a 3D reconstruction | |
| 650 | 4 | |a Generative adversarial network | |
| 650 | 4 | |a Partial convolution | |
| 650 | 4 | |a Ultrasound | |
| 700 | 1 | |a Fu, Tianyu |e verfasserin |4 aut | |
| 700 | 1 | |a Lin, Yucong |e verfasserin |4 aut | |
| 700 | 1 | |a Deng, Qiaoling |e verfasserin |4 aut | |
| 700 | 1 | |a Fan, Jingfan |e verfasserin |4 aut | |
| 700 | 1 | |a Song, Hong |e verfasserin |4 aut | |
| 700 | 1 | |a Cheng, Zhigang |e verfasserin |4 aut | |
| 700 | 1 | |a Liang, Ping |e verfasserin |4 aut | |
| 700 | 1 | |a Wang, Yongtian |e verfasserin |4 aut | |
| 700 | 1 | |a Yang, Jian |e verfasserin |4 aut | |
| 773 | 0 | 8 | |i Enthalten in |t Computer methods and programs in biomedicine |d 1993 |g 211(2021) vom: 25. Nov., Seite 106421 |w (DE-627)NLM012836133 |x 1872-7565 |7 nnns |
| 773 | 1 | 8 | |g volume:211 |g year:2021 |g day:25 |g month:11 |g pages:106421 |
| 856 | 4 | 0 | |u http://dx.doi.org/10.1016/j.cmpb.2021.106421 |3 Volltext |
| 912 | |a GBV_USEFLAG_A | ||
| 912 | |a GBV_NLM | ||
| 951 | |a AR | ||
| 952 | |d 211 |j 2021 |b 25 |c 11 |h 106421 | ||