A Bidirectional Feedforward Neural Network Architecture Using the Discretized Neural Memory Ordinary Differential Equation
Deep Feedforward Neural Networks (FNNs) with skip connections have revolutionized various image recognition tasks. In this paper, we propose a novel architecture called bidirectional FNN (BiFNN), which utilizes skip connections to aggregate features between its forward and backward paths. The BiFNN accepts any FNN as a plugin that can incorporate any general FNN model into its forward path, introducing only a few additional parameters in the cross-path connections. The backward path is implemented as a nonparameter layer, utilizing a discretized form of the neural memory Ordinary Differential Equation (nmODE), which is named [Formula: see text]-net. We provide a proof of convergence for the [Formula: see text]-net and evaluate its initial value problem. Our proposed architecture is evaluated on diverse image recognition datasets, including Fashion-MNIST, SVHN, CIFAR-10, CIFAR-100, and Tiny-ImageNet. The results demonstrate that BiFNNs offer significant improvements compared to embedded models such as ConvMixer, ResNet, ResNeXt, and Vision Transformer. Furthermore, BiFNNs can be fine-tuned to achieve comparable performance with embedded models on Tiny-ImageNet and ImageNet-1K datasets by loading the same pretrained parameters.
| Medienart: |
E-Artikel |
|---|
| Erscheinungsjahr: |
2024 |
|---|---|
| Erschienen: |
2024 |
| Enthalten in: |
Zur Gesamtaufnahme - volume:34 |
|---|---|
| Enthalten in: |
International journal of neural systems - 34(2024), 4 vom: 27. Feb., Seite 2450015 |
| Sprache: |
Englisch |
|---|
| Beteiligte Personen: |
Niu, Hao [VerfasserIn] |
|---|
| Links: |
|---|
| Themen: |
Bidirectional FNN |
|---|
| Anmerkungen: |
Date Completed 29.02.2024 Date Revised 29.02.2024 published: Print-Electronic Citation Status MEDLINE |
|---|
| doi: |
10.1142/S0129065724500151 |
|---|
| funding: |
|
|---|---|
| Förderinstitution / Projekttitel: |
|
| PPN (Katalog-ID): |
NLM368079759 |
|---|
| LEADER | 01000caa a22002652 4500 | ||
|---|---|---|---|
| 001 | NLM368079759 | ||
| 003 | DE-627 | ||
| 005 | 20240229234943.0 | ||
| 007 | cr uuu---uuuuu | ||
| 008 | 240206s2024 xx |||||o 00| ||eng c | ||
| 024 | 7 | |a 10.1142/S0129065724500151 |2 doi | |
| 028 | 5 | 2 | |a pubmed24n1311.xml |
| 035 | |a (DE-627)NLM368079759 | ||
| 035 | |a (NLM)38318709 | ||
| 040 | |a DE-627 |b ger |c DE-627 |e rakwb | ||
| 041 | |a eng | ||
| 100 | 1 | |a Niu, Hao |e verfasserin |4 aut | |
| 245 | 1 | 2 | |a A Bidirectional Feedforward Neural Network Architecture Using the Discretized Neural Memory Ordinary Differential Equation |
| 264 | 1 | |c 2024 | |
| 336 | |a Text |b txt |2 rdacontent | ||
| 337 | |a ƒaComputermedien |b c |2 rdamedia | ||
| 338 | |a ƒa Online-Ressource |b cr |2 rdacarrier | ||
| 500 | |a Date Completed 29.02.2024 | ||
| 500 | |a Date Revised 29.02.2024 | ||
| 500 | |a published: Print-Electronic | ||
| 500 | |a Citation Status MEDLINE | ||
| 520 | |a Deep Feedforward Neural Networks (FNNs) with skip connections have revolutionized various image recognition tasks. In this paper, we propose a novel architecture called bidirectional FNN (BiFNN), which utilizes skip connections to aggregate features between its forward and backward paths. The BiFNN accepts any FNN as a plugin that can incorporate any general FNN model into its forward path, introducing only a few additional parameters in the cross-path connections. The backward path is implemented as a nonparameter layer, utilizing a discretized form of the neural memory Ordinary Differential Equation (nmODE), which is named [Formula: see text]-net. We provide a proof of convergence for the [Formula: see text]-net and evaluate its initial value problem. Our proposed architecture is evaluated on diverse image recognition datasets, including Fashion-MNIST, SVHN, CIFAR-10, CIFAR-100, and Tiny-ImageNet. The results demonstrate that BiFNNs offer significant improvements compared to embedded models such as ConvMixer, ResNet, ResNeXt, and Vision Transformer. Furthermore, BiFNNs can be fine-tuned to achieve comparable performance with embedded models on Tiny-ImageNet and ImageNet-1K datasets by loading the same pretrained parameters | ||
| 650 | 4 | |a Journal Article | |
| 650 | 4 | |a Ordinary differential equation | |
| 650 | 4 | |a bidirectional FNN | |
| 650 | 4 | |a image recognition | |
| 650 | 4 | |a skip connection | |
| 700 | 1 | |a Yi, Zhang |e verfasserin |4 aut | |
| 700 | 1 | |a He, Tao |e verfasserin |4 aut | |
| 773 | 0 | 8 | |i Enthalten in |t International journal of neural systems |d 1993 |g 34(2024), 4 vom: 27. Feb., Seite 2450015 |w (DE-627)NLM074732498 |x 1793-6462 |7 nnns |
| 773 | 1 | 8 | |g volume:34 |g year:2024 |g number:4 |g day:27 |g month:02 |g pages:2450015 |
| 856 | 4 | 0 | |u http://dx.doi.org/10.1142/S0129065724500151 |3 Volltext |
| 912 | |a GBV_USEFLAG_A | ||
| 912 | |a GBV_NLM | ||
| 951 | |a AR | ||
| 952 | |d 34 |j 2024 |e 4 |b 27 |c 02 |h 2450015 | ||