Comparison of combination methods to create calibrated ensemble forecasts for seasonal influenza in the U.S
© 2023 The Authors. Statistics in Medicine published by John Wiley & Sons Ltd..
The characteristics of influenza seasons vary substantially from year to year, posing challenges for public health preparation and response. Influenza forecasting is used to inform seasonal outbreak response, which can in turn potentially reduce the impact of an epidemic. The United States Centers for Disease Control and Prevention, in collaboration with external researchers, has run an annual prospective influenza forecasting exercise, known as the FluSight challenge. Uniting theoretical results from the forecasting literature with domain-specific forecasts from influenza outbreaks, we applied parametric forecast combination methods that simultaneously optimize model weights and calibrate the ensemble via a beta transformation and made adjustments to the methods to reduce their complexity. We used the beta-transformed linear pool, the finite beta mixture model, and their equal weight adaptations to produce ensemble forecasts retrospectively for the 2016/2017, 2017/2018, and 2018/2019 influenza seasons in the U.S. We compared their performance to methods that were used in the FluSight challenge to produce the FluSight Network ensemble, namely the equally weighted linear pool and the linear pool. Ensemble forecasts produced from methods with a beta transformation were shown to outperform those from the equally weighted linear pool and the linear pool for all week-ahead targets across in the test seasons based on average log scores. We observed improvements in overall accuracy despite the beta-transformed linear pool or beta mixture methods' modest under-prediction across all targets and seasons. Combination techniques that explicitly adjust for known calibration issues in linear pooling should be considered to improve probabilistic scores in outbreak settings.
| Medienart: |
E-Artikel |
|---|
| Erscheinungsjahr: |
2023 |
|---|---|
| Erschienen: |
2023 |
| Enthalten in: |
Zur Gesamtaufnahme - volume:42 |
|---|---|
| Enthalten in: |
Statistics in medicine - 42(2023), 26 vom: 20. Nov., Seite 4696-4712 |
| Sprache: |
Englisch |
|---|
| Beteiligte Personen: |
Wattanachit, Nutcha [VerfasserIn] |
|---|
| Links: |
|---|
| Themen: |
Combination method |
|---|
| Anmerkungen: |
Date Completed 16.12.2023 Date Revised 30.01.2024 published: Print-Electronic Citation Status MEDLINE |
|---|
| doi: |
10.1002/sim.9884 |
|---|
| funding: |
|
|---|---|
| Förderinstitution / Projekttitel: |
|
| PPN (Katalog-ID): |
NLM361460813 |
|---|
| LEADER | 01000caa a22002652 4500 | ||
|---|---|---|---|
| 001 | NLM361460813 | ||
| 003 | DE-627 | ||
| 005 | 20240130232029.0 | ||
| 007 | cr uuu---uuuuu | ||
| 008 | 231226s2023 xx |||||o 00| ||eng c | ||
| 024 | 7 | |a 10.1002/sim.9884 |2 doi | |
| 028 | 5 | 2 | |a pubmed24n1275.xml |
| 035 | |a (DE-627)NLM361460813 | ||
| 035 | |a (NLM)37648218 | ||
| 040 | |a DE-627 |b ger |c DE-627 |e rakwb | ||
| 041 | |a eng | ||
| 100 | 1 | |a Wattanachit, Nutcha |e verfasserin |4 aut | |
| 245 | 1 | 0 | |a Comparison of combination methods to create calibrated ensemble forecasts for seasonal influenza in the U.S |
| 264 | 1 | |c 2023 | |
| 336 | |a Text |b txt |2 rdacontent | ||
| 337 | |a ƒaComputermedien |b c |2 rdamedia | ||
| 338 | |a ƒa Online-Ressource |b cr |2 rdacarrier | ||
| 500 | |a Date Completed 16.12.2023 | ||
| 500 | |a Date Revised 30.01.2024 | ||
| 500 | |a published: Print-Electronic | ||
| 500 | |a Citation Status MEDLINE | ||
| 520 | |a © 2023 The Authors. Statistics in Medicine published by John Wiley & Sons Ltd. | ||
| 520 | |a The characteristics of influenza seasons vary substantially from year to year, posing challenges for public health preparation and response. Influenza forecasting is used to inform seasonal outbreak response, which can in turn potentially reduce the impact of an epidemic. The United States Centers for Disease Control and Prevention, in collaboration with external researchers, has run an annual prospective influenza forecasting exercise, known as the FluSight challenge. Uniting theoretical results from the forecasting literature with domain-specific forecasts from influenza outbreaks, we applied parametric forecast combination methods that simultaneously optimize model weights and calibrate the ensemble via a beta transformation and made adjustments to the methods to reduce their complexity. We used the beta-transformed linear pool, the finite beta mixture model, and their equal weight adaptations to produce ensemble forecasts retrospectively for the 2016/2017, 2017/2018, and 2018/2019 influenza seasons in the U.S. We compared their performance to methods that were used in the FluSight challenge to produce the FluSight Network ensemble, namely the equally weighted linear pool and the linear pool. Ensemble forecasts produced from methods with a beta transformation were shown to outperform those from the equally weighted linear pool and the linear pool for all week-ahead targets across in the test seasons based on average log scores. We observed improvements in overall accuracy despite the beta-transformed linear pool or beta mixture methods' modest under-prediction across all targets and seasons. Combination techniques that explicitly adjust for known calibration issues in linear pooling should be considered to improve probabilistic scores in outbreak settings | ||
| 650 | 4 | |a Journal Article | |
| 650 | 4 | |a combination method | |
| 650 | 4 | |a ensemble | |
| 650 | 4 | |a epidemiology | |
| 650 | 4 | |a infectious disease forecasting | |
| 650 | 4 | |a seasonal influenza | |
| 700 | 1 | |a Ray, Evan L |e verfasserin |4 aut | |
| 700 | 1 | |a McAndrew, Thomas C |e verfasserin |4 aut | |
| 700 | 1 | |a Reich, Nicholas G |e verfasserin |4 aut | |
| 773 | 0 | 8 | |i Enthalten in |t Statistics in medicine |d 1984 |g 42(2023), 26 vom: 20. Nov., Seite 4696-4712 |w (DE-627)NLM012664596 |x 1097-0258 |7 nnns |
| 773 | 1 | 8 | |g volume:42 |g year:2023 |g number:26 |g day:20 |g month:11 |g pages:4696-4712 |
| 856 | 4 | 0 | |u http://dx.doi.org/10.1002/sim.9884 |3 Volltext |
| 912 | |a GBV_USEFLAG_A | ||
| 912 | |a GBV_NLM | ||
| 951 | |a AR | ||
| 952 | |d 42 |j 2023 |e 26 |b 20 |c 11 |h 4696-4712 | ||