Few-shot biomedical image segmentation using diffusion models : Beyond image generation
Copyright © 2023 Elsevier B.V. All rights reserved..
BACKGROUND: Medical image analysis pipelines often involve segmentation, which requires a large amount of annotated training data, which is time-consuming and costly. To address this issue, we proposed leveraging generative models to achieve few-shot image segmentation.
METHODS: We trained a denoising diffusion probabilistic model (DDPM) on 480,407 pelvis radiographs to generate 256 ✕ 256 px synthetic images. The DDPM was conditioned on demographic and radiologic characteristics and was rigorously validated by domain experts and objective image quality metrics (Frechet inception distance [FID] and inception score [IS]). For the next step, three landmarks (greater trochanter [GT], lesser trochanter [LT], and obturator foramen [OF]) were annotated on 45 real-patient radiographs; 25 for training and 20 for testing. To extract features, each image was passed through the pre-trained DDPM at three timesteps and for each pass, features from specific blocks were extracted. The features were concatenated with the real image to form an image with 4225 channels. The feature-set was broken into random patches, which were fed to a U-Net. Dice Similarity Coefficient (DSC) was used to compare the performance with a vanilla U-Net trained on radiographs.
RESULTS: Expert accuracy was 57.5 % in determining real versus generated images, while the model reached an FID = 7.2 and IS = 210. The segmentation UNet trained on the 20 feature-sets achieved a DSC of 0.90, 0.84, and 0.61 for OF, GT, and LT segmentation, respectively, which was at least 0.30 points higher than the naively trained model.
CONCLUSION: We demonstrated the applicability of DDPMs as feature extractors, facilitating medical image segmentation with few annotated samples.
| Medienart: |
E-Artikel |
|---|
| Erscheinungsjahr: |
2023 |
|---|---|
| Erschienen: |
2023 |
| Enthalten in: |
Zur Gesamtaufnahme - volume:242 |
|---|---|
| Enthalten in: |
Computer methods and programs in biomedicine - 242(2023) vom: 08. Dez., Seite 107832 |
| Sprache: |
Englisch |
|---|
| Beteiligte Personen: |
Khosravi, Bardia [VerfasserIn] |
|---|
| Links: |
|---|
| Themen: |
10X0709Y6I |
|---|
| Anmerkungen: |
Date Completed 14.11.2023 Date Revised 14.11.2023 published: Print-Electronic Citation Status MEDLINE |
|---|
| doi: |
10.1016/j.cmpb.2023.107832 |
|---|
| funding: |
|
|---|---|
| Förderinstitution / Projekttitel: |
|
| PPN (Katalog-ID): |
NLM362740208 |
|---|
| LEADER | 01000naa a22002652 4500 | ||
|---|---|---|---|
| 001 | NLM362740208 | ||
| 003 | DE-627 | ||
| 005 | 20231226091854.0 | ||
| 007 | cr uuu---uuuuu | ||
| 008 | 231226s2023 xx |||||o 00| ||eng c | ||
| 024 | 7 | |a 10.1016/j.cmpb.2023.107832 |2 doi | |
| 028 | 5 | 2 | |a pubmed24n1209.xml |
| 035 | |a (DE-627)NLM362740208 | ||
| 035 | |a (NLM)37778140 | ||
| 035 | |a (PII)S0169-2607(23)00498-4 | ||
| 040 | |a DE-627 |b ger |c DE-627 |e rakwb | ||
| 041 | |a eng | ||
| 100 | 1 | |a Khosravi, Bardia |e verfasserin |4 aut | |
| 245 | 1 | 0 | |a Few-shot biomedical image segmentation using diffusion models |b Beyond image generation |
| 264 | 1 | |c 2023 | |
| 336 | |a Text |b txt |2 rdacontent | ||
| 337 | |a ƒaComputermedien |b c |2 rdamedia | ||
| 338 | |a ƒa Online-Ressource |b cr |2 rdacarrier | ||
| 500 | |a Date Completed 14.11.2023 | ||
| 500 | |a Date Revised 14.11.2023 | ||
| 500 | |a published: Print-Electronic | ||
| 500 | |a Citation Status MEDLINE | ||
| 520 | |a Copyright © 2023 Elsevier B.V. All rights reserved. | ||
| 520 | |a BACKGROUND: Medical image analysis pipelines often involve segmentation, which requires a large amount of annotated training data, which is time-consuming and costly. To address this issue, we proposed leveraging generative models to achieve few-shot image segmentation | ||
| 520 | |a METHODS: We trained a denoising diffusion probabilistic model (DDPM) on 480,407 pelvis radiographs to generate 256 ✕ 256 px synthetic images. The DDPM was conditioned on demographic and radiologic characteristics and was rigorously validated by domain experts and objective image quality metrics (Frechet inception distance [FID] and inception score [IS]). For the next step, three landmarks (greater trochanter [GT], lesser trochanter [LT], and obturator foramen [OF]) were annotated on 45 real-patient radiographs; 25 for training and 20 for testing. To extract features, each image was passed through the pre-trained DDPM at three timesteps and for each pass, features from specific blocks were extracted. The features were concatenated with the real image to form an image with 4225 channels. The feature-set was broken into random patches, which were fed to a U-Net. Dice Similarity Coefficient (DSC) was used to compare the performance with a vanilla U-Net trained on radiographs | ||
| 520 | |a RESULTS: Expert accuracy was 57.5 % in determining real versus generated images, while the model reached an FID = 7.2 and IS = 210. The segmentation UNet trained on the 20 feature-sets achieved a DSC of 0.90, 0.84, and 0.61 for OF, GT, and LT segmentation, respectively, which was at least 0.30 points higher than the naively trained model | ||
| 520 | |a CONCLUSION: We demonstrated the applicability of DDPMs as feature extractors, facilitating medical image segmentation with few annotated samples | ||
| 650 | 4 | |a Journal Article | |
| 650 | 4 | |a Diffusion models | |
| 650 | 4 | |a Generative AI | |
| 650 | 4 | |a Orthopedics surgery | |
| 650 | 4 | |a Pelvis radiographs | |
| 650 | 4 | |a Semantic segmentation | |
| 650 | 4 | |a Synthetic data | |
| 650 | 7 | |a dihydroxydiphenyl-pyridyl methane |2 NLM | |
| 650 | 7 | |a R09078E41Y |2 NLM | |
| 650 | 7 | |a Bisacodyl |2 NLM | |
| 650 | 7 | |a 10X0709Y6I |2 NLM | |
| 700 | 1 | |a Rouzrokh, Pouria |e verfasserin |4 aut | |
| 700 | 1 | |a Mickley, John P |e verfasserin |4 aut | |
| 700 | 1 | |a Faghani, Shahriar |e verfasserin |4 aut | |
| 700 | 1 | |a Mulford, Kellen |e verfasserin |4 aut | |
| 700 | 1 | |a Yang, Linjun |e verfasserin |4 aut | |
| 700 | 1 | |a Larson, A Noelle |e verfasserin |4 aut | |
| 700 | 1 | |a Howe, Benjamin M |e verfasserin |4 aut | |
| 700 | 1 | |a Erickson, Bradley J |e verfasserin |4 aut | |
| 700 | 1 | |a Taunton, Michael J |e verfasserin |4 aut | |
| 700 | 1 | |a Wyles, Cody C |e verfasserin |4 aut | |
| 773 | 0 | 8 | |i Enthalten in |t Computer methods and programs in biomedicine |d 1993 |g 242(2023) vom: 08. Dez., Seite 107832 |w (DE-627)NLM012836133 |x 1872-7565 |7 nnns |
| 773 | 1 | 8 | |g volume:242 |g year:2023 |g day:08 |g month:12 |g pages:107832 |
| 856 | 4 | 0 | |u http://dx.doi.org/10.1016/j.cmpb.2023.107832 |3 Volltext |
| 912 | |a GBV_USEFLAG_A | ||
| 912 | |a GBV_NLM | ||
| 951 | |a AR | ||
| 952 | |d 242 |j 2023 |b 08 |c 12 |h 107832 | ||