Accuracy of real-time multi-model ensemble forecasts for seasonal influenza in the U.S.
Seasonal influenza results in substantial annual morbidity and mortality in the United States and worldwide. Accurate forecasts of key features of influenza epidemics, such as the timing and severity of the peak incidence in a given season, can inform public health response to outbreaks. As part of...
| Main Authors: | , , , , , , , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Public Library of Science (PLoS)
2019-11-01
|
| Series: | PLoS Computational Biology |
| Online Access: | https://doi.org/10.1371/journal.pcbi.1007486 |
| _version_ | 1819259652772200448 |
|---|---|
| author | Nicholas G Reich Craig J McGowan Teresa K Yamana Abhinav Tushar Evan L Ray Dave Osthus Sasikiran Kandula Logan C Brooks Willow Crawford-Crudell Graham Casey Gibson Evan Moore Rebecca Silva Matthew Biggerstaff Michael A Johansson Roni Rosenfeld Jeffrey Shaman |
| author_facet | Nicholas G Reich Craig J McGowan Teresa K Yamana Abhinav Tushar Evan L Ray Dave Osthus Sasikiran Kandula Logan C Brooks Willow Crawford-Crudell Graham Casey Gibson Evan Moore Rebecca Silva Matthew Biggerstaff Michael A Johansson Roni Rosenfeld Jeffrey Shaman |
| author_sort | Nicholas G Reich |
| collection | DOAJ |
| description | Seasonal influenza results in substantial annual morbidity and mortality in the United States and worldwide. Accurate forecasts of key features of influenza epidemics, such as the timing and severity of the peak incidence in a given season, can inform public health response to outbreaks. As part of ongoing efforts to incorporate data and advanced analytical methods into public health decision-making, the United States Centers for Disease Control and Prevention (CDC) has organized seasonal influenza forecasting challenges since the 2013/2014 season. In the 2017/2018 season, 22 teams participated. A subset of four teams created a research consortium called the FluSight Network in early 2017. During the 2017/2018 season they worked together to produce a collaborative multi-model ensemble that combined 21 separate component models into a single model using a machine learning technique called stacking. This approach creates a weighted average of predictive densities where the weight for each component is determined by maximizing overall ensemble accuracy over past seasons. In the 2017/2018 influenza season, one of the largest seasonal outbreaks in the last 15 years, this multi-model ensemble performed better on average than all individual component models and placed second overall in the CDC challenge. It also outperformed the baseline multi-model ensemble created by the CDC that took a simple average of all models submitted to the forecasting challenge. This project shows that collaborative efforts between research teams to develop ensemble forecasting approaches can bring measurable improvements in forecast accuracy and important reductions in the variability of performance from year to year. Efforts such as this, that emphasize real-time testing and evaluation of forecasting models and facilitate the close collaboration between public health officials and modeling researchers, are essential to improving our understanding of how best to use forecasts to improve public health response to seasonal and emerging epidemic threats. |
| first_indexed | 2024-12-23T19:13:25Z |
| format | Article |
| id | doaj.art-8fddc76ce8384bd69afafa34f538fa07 |
| institution | Directory Open Access Journal |
| issn | 1553-734X 1553-7358 |
| language | English |
| last_indexed | 2024-12-23T19:13:25Z |
| publishDate | 2019-11-01 |
| publisher | Public Library of Science (PLoS) |
| record_format | Article |
| series | PLoS Computational Biology |
| spelling | doaj.art-8fddc76ce8384bd69afafa34f538fa072022-12-21T17:34:23ZengPublic Library of Science (PLoS)PLoS Computational Biology1553-734X1553-73582019-11-011511e100748610.1371/journal.pcbi.1007486Accuracy of real-time multi-model ensemble forecasts for seasonal influenza in the U.S.Nicholas G ReichCraig J McGowanTeresa K YamanaAbhinav TusharEvan L RayDave OsthusSasikiran KandulaLogan C BrooksWillow Crawford-CrudellGraham Casey GibsonEvan MooreRebecca SilvaMatthew BiggerstaffMichael A JohanssonRoni RosenfeldJeffrey ShamanSeasonal influenza results in substantial annual morbidity and mortality in the United States and worldwide. Accurate forecasts of key features of influenza epidemics, such as the timing and severity of the peak incidence in a given season, can inform public health response to outbreaks. As part of ongoing efforts to incorporate data and advanced analytical methods into public health decision-making, the United States Centers for Disease Control and Prevention (CDC) has organized seasonal influenza forecasting challenges since the 2013/2014 season. In the 2017/2018 season, 22 teams participated. A subset of four teams created a research consortium called the FluSight Network in early 2017. During the 2017/2018 season they worked together to produce a collaborative multi-model ensemble that combined 21 separate component models into a single model using a machine learning technique called stacking. This approach creates a weighted average of predictive densities where the weight for each component is determined by maximizing overall ensemble accuracy over past seasons. In the 2017/2018 influenza season, one of the largest seasonal outbreaks in the last 15 years, this multi-model ensemble performed better on average than all individual component models and placed second overall in the CDC challenge. It also outperformed the baseline multi-model ensemble created by the CDC that took a simple average of all models submitted to the forecasting challenge. This project shows that collaborative efforts between research teams to develop ensemble forecasting approaches can bring measurable improvements in forecast accuracy and important reductions in the variability of performance from year to year. Efforts such as this, that emphasize real-time testing and evaluation of forecasting models and facilitate the close collaboration between public health officials and modeling researchers, are essential to improving our understanding of how best to use forecasts to improve public health response to seasonal and emerging epidemic threats.https://doi.org/10.1371/journal.pcbi.1007486 |
| spellingShingle | Nicholas G Reich Craig J McGowan Teresa K Yamana Abhinav Tushar Evan L Ray Dave Osthus Sasikiran Kandula Logan C Brooks Willow Crawford-Crudell Graham Casey Gibson Evan Moore Rebecca Silva Matthew Biggerstaff Michael A Johansson Roni Rosenfeld Jeffrey Shaman Accuracy of real-time multi-model ensemble forecasts for seasonal influenza in the U.S. PLoS Computational Biology |
| title | Accuracy of real-time multi-model ensemble forecasts for seasonal influenza in the U.S. |
| title_full | Accuracy of real-time multi-model ensemble forecasts for seasonal influenza in the U.S. |
| title_fullStr | Accuracy of real-time multi-model ensemble forecasts for seasonal influenza in the U.S. |
| title_full_unstemmed | Accuracy of real-time multi-model ensemble forecasts for seasonal influenza in the U.S. |
| title_short | Accuracy of real-time multi-model ensemble forecasts for seasonal influenza in the U.S. |
| title_sort | accuracy of real time multi model ensemble forecasts for seasonal influenza in the u s |
| url | https://doi.org/10.1371/journal.pcbi.1007486 |
| work_keys_str_mv | AT nicholasgreich accuracyofrealtimemultimodelensembleforecastsforseasonalinfluenzaintheus AT craigjmcgowan accuracyofrealtimemultimodelensembleforecastsforseasonalinfluenzaintheus AT teresakyamana accuracyofrealtimemultimodelensembleforecastsforseasonalinfluenzaintheus AT abhinavtushar accuracyofrealtimemultimodelensembleforecastsforseasonalinfluenzaintheus AT evanlray accuracyofrealtimemultimodelensembleforecastsforseasonalinfluenzaintheus AT daveosthus accuracyofrealtimemultimodelensembleforecastsforseasonalinfluenzaintheus AT sasikirankandula accuracyofrealtimemultimodelensembleforecastsforseasonalinfluenzaintheus AT logancbrooks accuracyofrealtimemultimodelensembleforecastsforseasonalinfluenzaintheus AT willowcrawfordcrudell accuracyofrealtimemultimodelensembleforecastsforseasonalinfluenzaintheus AT grahamcaseygibson accuracyofrealtimemultimodelensembleforecastsforseasonalinfluenzaintheus AT evanmoore accuracyofrealtimemultimodelensembleforecastsforseasonalinfluenzaintheus AT rebeccasilva accuracyofrealtimemultimodelensembleforecastsforseasonalinfluenzaintheus AT matthewbiggerstaff accuracyofrealtimemultimodelensembleforecastsforseasonalinfluenzaintheus AT michaelajohansson accuracyofrealtimemultimodelensembleforecastsforseasonalinfluenzaintheus AT ronirosenfeld accuracyofrealtimemultimodelensembleforecastsforseasonalinfluenzaintheus AT jeffreyshaman accuracyofrealtimemultimodelensembleforecastsforseasonalinfluenzaintheus |