Generative interpolation and restoration of images using deep learning for improved 3D tissue mapping
The development of novel imaging platforms has improved our ability to collect and analyze large three-dimensional (3D) biological imaging datasets. Advances in computing have led to an ability to extract complex spatial information from these data, such as the composition, morphology, and interactions of multi-cellular structures, rare events, and integration of multi-modal features combining anatomical, molecular, and transcriptomic (among other) information. Yet, the accuracy of these quantitative results is intrinsically limited by the quality of the input images, which can contain missing or damaged regions, or can be of poor resolution due to mechanical, temporal, or financial constraints. In applications ranging from intact imaging (e.g. light-sheet microscopy and magnetic resonance imaging) to sectioning based platforms (e.g. serial histology and serial section transmission electron microscopy), the quality and resolution of imaging data has become paramount. Here, we address these challenges by leveraging frame interpolation for large image motion (FILM), a generative AI model originally developed for temporal interpolation, for spatial interpolation of a range of 3D image types. Comparative analysis demonstrates the superiority of FILM over traditional linear interpolation to produce functional synthetic images, due to its ability to better preserve biological information including microanatomical features and cell counts, as well as image quality, such as contrast, variance, and luminance. FILM repairs tissue damages in images and reduces stitching artifacts. We show that FILM can decrease imaging time by synthesizing skipped images. We demonstrate the versatility of our method with a wide range of imaging modalities (histology, tissue-clearing/light-sheet microscopy, magnetic resonance imaging, serial section transmission electron microscopy), species (human, mouse), healthy and diseased tissues (pancreas, lung, brain), staining techniques (IHC, H&E), and pixel resolutions (8 nm, 2 μm, 1mm). Overall, we demonstrate the potential of generative AI in improving the resolution, throughput, and quality of biological image datasets, enabling improved 3D imaging.
| Medienart: |
E-Artikel |
|---|
| Erscheinungsjahr: |
2024 |
|---|---|
| Erschienen: |
2024 |
| Enthalten in: |
Zur Gesamtaufnahme - year:2024 |
|---|---|
| Enthalten in: |
bioRxiv : the preprint server for biology - (2024) vom: 28. März |
| Sprache: |
Englisch |
|---|
| Beteiligte Personen: |
Joshi, Saurabh [VerfasserIn] |
|---|
| Links: |
|---|
| Themen: |
|---|
| Anmerkungen: |
Date Revised 25.04.2024 published: Electronic Citation Status PubMed-not-MEDLINE |
|---|
| doi: |
10.1101/2024.03.07.583909 |
|---|
| funding: |
|
|---|---|
| Förderinstitution / Projekttitel: |
|
| PPN (Katalog-ID): |
NLM36986199X |
|---|
| LEADER | 01000caa a22002652 4500 | ||
|---|---|---|---|
| 001 | NLM36986199X | ||
| 003 | DE-627 | ||
| 005 | 20240425233205.0 | ||
| 007 | cr uuu---uuuuu | ||
| 008 | 240318s2024 xx |||||o 00| ||eng c | ||
| 024 | 7 | |a 10.1101/2024.03.07.583909 |2 doi | |
| 028 | 5 | 2 | |a pubmed24n1386.xml |
| 035 | |a (DE-627)NLM36986199X | ||
| 035 | |a (NLM)38496512 | ||
| 035 | |a (PII)2024.03.07.583909 | ||
| 040 | |a DE-627 |b ger |c DE-627 |e rakwb | ||
| 041 | |a eng | ||
| 100 | 1 | |a Joshi, Saurabh |e verfasserin |4 aut | |
| 245 | 1 | 0 | |a Generative interpolation and restoration of images using deep learning for improved 3D tissue mapping |
| 264 | 1 | |c 2024 | |
| 336 | |a Text |b txt |2 rdacontent | ||
| 337 | |a ƒaComputermedien |b c |2 rdamedia | ||
| 338 | |a ƒa Online-Ressource |b cr |2 rdacarrier | ||
| 500 | |a Date Revised 25.04.2024 | ||
| 500 | |a published: Electronic | ||
| 500 | |a Citation Status PubMed-not-MEDLINE | ||
| 520 | |a The development of novel imaging platforms has improved our ability to collect and analyze large three-dimensional (3D) biological imaging datasets. Advances in computing have led to an ability to extract complex spatial information from these data, such as the composition, morphology, and interactions of multi-cellular structures, rare events, and integration of multi-modal features combining anatomical, molecular, and transcriptomic (among other) information. Yet, the accuracy of these quantitative results is intrinsically limited by the quality of the input images, which can contain missing or damaged regions, or can be of poor resolution due to mechanical, temporal, or financial constraints. In applications ranging from intact imaging (e.g. light-sheet microscopy and magnetic resonance imaging) to sectioning based platforms (e.g. serial histology and serial section transmission electron microscopy), the quality and resolution of imaging data has become paramount. Here, we address these challenges by leveraging frame interpolation for large image motion (FILM), a generative AI model originally developed for temporal interpolation, for spatial interpolation of a range of 3D image types. Comparative analysis demonstrates the superiority of FILM over traditional linear interpolation to produce functional synthetic images, due to its ability to better preserve biological information including microanatomical features and cell counts, as well as image quality, such as contrast, variance, and luminance. FILM repairs tissue damages in images and reduces stitching artifacts. We show that FILM can decrease imaging time by synthesizing skipped images. We demonstrate the versatility of our method with a wide range of imaging modalities (histology, tissue-clearing/light-sheet microscopy, magnetic resonance imaging, serial section transmission electron microscopy), species (human, mouse), healthy and diseased tissues (pancreas, lung, brain), staining techniques (IHC, H&E), and pixel resolutions (8 nm, 2 μm, 1mm). Overall, we demonstrate the potential of generative AI in improving the resolution, throughput, and quality of biological image datasets, enabling improved 3D imaging | ||
| 650 | 4 | |a Preprint | |
| 700 | 1 | |a Forjaz, André |e verfasserin |4 aut | |
| 700 | 1 | |a Han, Kyu Sang |e verfasserin |4 aut | |
| 700 | 1 | |a Shen, Yu |e verfasserin |4 aut | |
| 700 | 1 | |a Queiroga, Vasco |e verfasserin |4 aut | |
| 700 | 1 | |a Xenes, Daniel |e verfasserin |4 aut | |
| 700 | 1 | |a Matelsk, Jordan |e verfasserin |4 aut | |
| 700 | 1 | |a Wester, Brock |e verfasserin |4 aut | |
| 700 | 1 | |a Barrutia, Arrate Munoz |e verfasserin |4 aut | |
| 700 | 1 | |a Kiemen, Ashley L |e verfasserin |4 aut | |
| 700 | 1 | |a Wu, Pei-Hsun |e verfasserin |4 aut | |
| 700 | 1 | |a Wirtz, Denis |e verfasserin |4 aut | |
| 773 | 0 | 8 | |i Enthalten in |t bioRxiv : the preprint server for biology |d 2020 |g (2024) vom: 28. März |w (DE-627)NLM31090014X |7 nnns |
| 773 | 1 | 8 | |g year:2024 |g day:28 |g month:03 |
| 856 | 4 | 0 | |u http://dx.doi.org/10.1101/2024.03.07.583909 |3 Volltext |
| 912 | |a GBV_USEFLAG_A | ||
| 912 | |a GBV_NLM | ||
| 951 | |a AR | ||
| 952 | |j 2024 |b 28 |c 03 | ||