Spatial multi-attention conditional neural processes
Copyright © 2024 Elsevier Ltd. All rights reserved..
Spatial prediction tasks are challenging when observed samples are sparse and prediction samples are abundant. Gaussian processes (GPs) are commonly used in spatial prediction tasks and have the advantage of measuring the uncertainty of the interpolation result. However, as the sample size increases, GPs suffer from significant overhead. Standard neural networks (NNs) provide a powerful and scalable solution for modeling spatial data, but they often overfit small sample data. Based on conditional neural processes (CNPs), which combine the advantages of GPs and NNs, we propose a new framework called Spatial Multi-Attention Conditional Neural Processes (SMACNPs) for spatial small sample prediction tasks. SMACNPs are a modular model that can predict targets by employing different attention mechanisms to extract relevant information from different forms of sample data. The task representation is inferred by measuring the spatial correlation contained in different sample points and the relationship contained in attribute variables, respectively. The distribution of the target variable is predicted by GPs parameterized by NNs. SMACNPs allow us to obtain accurate predictions of the target value while quantifying the prediction uncertainty. Experiments on spatial prediction tasks on simulated and real-world datasets demonstrate that this framework flexibly incorporates spatial context and correlation into the model, achieving state-of-the-art results in spatial small sample prediction tasks in terms of both predictive performance and reliability. For example, on the California housing dataset, our method reduces MAE by 8% and MSE by 7% compared to the second-best method. In addition, a spatiotemporal prediction task to forecast traffic speed further confirms the effectiveness and generality of our method.
Medienart: |
E-Artikel |
---|
Erscheinungsjahr: |
2024 |
---|---|
Erschienen: |
2024 |
Enthalten in: |
Zur Gesamtaufnahme - volume:173 |
---|---|
Enthalten in: |
Neural networks : the official journal of the International Neural Network Society - 173(2024) vom: 25. März, Seite 106201 |
Sprache: |
Englisch |
---|
Beteiligte Personen: |
Bao, Li-Li [VerfasserIn] |
---|
Links: |
---|
Themen: |
Attention mechanism |
---|
Anmerkungen: |
Date Completed 26.03.2024 Date Revised 26.03.2024 published: Print-Electronic Citation Status MEDLINE |
---|
doi: |
10.1016/j.neunet.2024.106201 |
---|
funding: |
|
---|---|
Förderinstitution / Projekttitel: |
|
PPN (Katalog-ID): |
NLM369370724 |
---|
LEADER | 01000caa a22002652 4500 | ||
---|---|---|---|
001 | NLM369370724 | ||
003 | DE-627 | ||
005 | 20240326235721.0 | ||
007 | cr uuu---uuuuu | ||
008 | 240307s2024 xx |||||o 00| ||eng c | ||
024 | 7 | |a 10.1016/j.neunet.2024.106201 |2 doi | |
028 | 5 | 2 | |a pubmed24n1349.xml |
035 | |a (DE-627)NLM369370724 | ||
035 | |a (NLM)38447305 | ||
035 | |a (PII)S0893-6080(24)00125-4 | ||
040 | |a DE-627 |b ger |c DE-627 |e rakwb | ||
041 | |a eng | ||
100 | 1 | |a Bao, Li-Li |e verfasserin |4 aut | |
245 | 1 | 0 | |a Spatial multi-attention conditional neural processes |
264 | 1 | |c 2024 | |
336 | |a Text |b txt |2 rdacontent | ||
337 | |a ƒaComputermedien |b c |2 rdamedia | ||
338 | |a ƒa Online-Ressource |b cr |2 rdacarrier | ||
500 | |a Date Completed 26.03.2024 | ||
500 | |a Date Revised 26.03.2024 | ||
500 | |a published: Print-Electronic | ||
500 | |a Citation Status MEDLINE | ||
520 | |a Copyright © 2024 Elsevier Ltd. All rights reserved. | ||
520 | |a Spatial prediction tasks are challenging when observed samples are sparse and prediction samples are abundant. Gaussian processes (GPs) are commonly used in spatial prediction tasks and have the advantage of measuring the uncertainty of the interpolation result. However, as the sample size increases, GPs suffer from significant overhead. Standard neural networks (NNs) provide a powerful and scalable solution for modeling spatial data, but they often overfit small sample data. Based on conditional neural processes (CNPs), which combine the advantages of GPs and NNs, we propose a new framework called Spatial Multi-Attention Conditional Neural Processes (SMACNPs) for spatial small sample prediction tasks. SMACNPs are a modular model that can predict targets by employing different attention mechanisms to extract relevant information from different forms of sample data. The task representation is inferred by measuring the spatial correlation contained in different sample points and the relationship contained in attribute variables, respectively. The distribution of the target variable is predicted by GPs parameterized by NNs. SMACNPs allow us to obtain accurate predictions of the target value while quantifying the prediction uncertainty. Experiments on spatial prediction tasks on simulated and real-world datasets demonstrate that this framework flexibly incorporates spatial context and correlation into the model, achieving state-of-the-art results in spatial small sample prediction tasks in terms of both predictive performance and reliability. For example, on the California housing dataset, our method reduces MAE by 8% and MSE by 7% compared to the second-best method. In addition, a spatiotemporal prediction task to forecast traffic speed further confirms the effectiveness and generality of our method | ||
650 | 4 | |a Journal Article | |
650 | 4 | |a Attention mechanism | |
650 | 4 | |a Conditional neural processes | |
650 | 4 | |a Gaussian processes | |
650 | 4 | |a Spatial prediction | |
700 | 1 | |a Zhang, Jiang-She |e verfasserin |4 aut | |
700 | 1 | |a Zhang, Chun-Xia |e verfasserin |4 aut | |
773 | 0 | 8 | |i Enthalten in |t Neural networks : the official journal of the International Neural Network Society |d 1996 |g 173(2024) vom: 25. März, Seite 106201 |w (DE-627)NLM087746824 |x 1879-2782 |7 nnns |
773 | 1 | 8 | |g volume:173 |g year:2024 |g day:25 |g month:03 |g pages:106201 |
856 | 4 | 0 | |u http://dx.doi.org/10.1016/j.neunet.2024.106201 |3 Volltext |
912 | |a GBV_USEFLAG_A | ||
912 | |a GBV_NLM | ||
951 | |a AR | ||
952 | |d 173 |j 2024 |b 25 |c 03 |h 106201 |