Retrospective evaluation of real-time estimates of global COVID-19 transmission trends and mortality forecasts
Since 8<sup>th</sup> March 2020 up to the time of writing, we have been producing near real-time weekly estimates of SARS-CoV-2 transmissibility and forecasts of deaths due to COVID-19 for all countries with evidence of sustained transmission, shared online. We also developed a novel heu...
| Main Authors: | , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Journal article |
| Language: | English |
| Published: |
Public Library of Science
2023
|
| _version_ | 1797111273521938432 |
|---|---|
| author | Bhatia, S Parag, KV Wardle, J Nash, RK Imai, N Van Elsland, SL Lassmann, B Brownstein, JS Desai, A Herringer, M Sewalk, K Loeb, SC Ramatowski, J Cuomo-Dannenburg, G Jauneikaite, E Unwin, HJT Riley, S Ferguson, N Donnelly, C Cori, A Nouvellet, P |
| author_facet | Bhatia, S Parag, KV Wardle, J Nash, RK Imai, N Van Elsland, SL Lassmann, B Brownstein, JS Desai, A Herringer, M Sewalk, K Loeb, SC Ramatowski, J Cuomo-Dannenburg, G Jauneikaite, E Unwin, HJT Riley, S Ferguson, N Donnelly, C Cori, A Nouvellet, P |
| author_sort | Bhatia, S |
| collection | OXFORD |
| description | Since 8<sup>th</sup> March 2020 up to the time of writing, we have been producing near real-time weekly estimates of SARS-CoV-2 transmissibility and forecasts of deaths due to COVID-19 for all countries with evidence of sustained transmission, shared online. We also developed a novel heuristic to combine weekly estimates of transmissibility to produce forecasts over a 4-week horizon. Here we present a retrospective evaluation of the forecasts produced between 8<sup>th</sup> March to 29<sup>th</sup> November 2020 for 81 countries. We evaluated the robustness of the forecasts produced in real-time using relative error, coverage probability, and comparisons with null models. During the 39-week period covered by this study, both the short- and medium-term forecasts captured well the epidemic trajectory across different waves of COVID-19 infections with small relative errors over the forecast horizon. The model was well calibrated with 56.3% and 45.6% of the observations lying in the 50% Credible Interval in 1-week and 4-week ahead forecasts respectively. The retrospective evaluation of our models shows that simple transmission models calibrated using routine disease surveillance data can reliably capture the epidemic trajectory in multiple countries. The medium-term forecasts can be used in conjunction with the short-term forecasts of COVID-19 mortality as a useful planning tool as countries continue to relax public health measures. |
| first_indexed | 2024-03-07T08:06:26Z |
| format | Journal article |
| id | oxford-uuid:53017192-fff1-426b-840f-e59b6e922b93 |
| institution | University of Oxford |
| language | English |
| last_indexed | 2024-03-07T08:06:26Z |
| publishDate | 2023 |
| publisher | Public Library of Science |
| record_format | dspace |
| spelling | oxford-uuid:53017192-fff1-426b-840f-e59b6e922b932023-11-06T12:17:29ZRetrospective evaluation of real-time estimates of global COVID-19 transmission trends and mortality forecastsJournal articlehttp://purl.org/coar/resource_type/c_dcae04bcuuid:53017192-fff1-426b-840f-e59b6e922b93EnglishSymplectic ElementsPublic Library of Science2023Bhatia, SParag, KVWardle, JNash, RKImai, NVan Elsland, SLLassmann, BBrownstein, JSDesai, AHerringer, MSewalk, KLoeb, SCRamatowski, JCuomo-Dannenburg, GJauneikaite, EUnwin, HJTRiley, SFerguson, NDonnelly, CCori, ANouvellet, PSince 8<sup>th</sup> March 2020 up to the time of writing, we have been producing near real-time weekly estimates of SARS-CoV-2 transmissibility and forecasts of deaths due to COVID-19 for all countries with evidence of sustained transmission, shared online. We also developed a novel heuristic to combine weekly estimates of transmissibility to produce forecasts over a 4-week horizon. Here we present a retrospective evaluation of the forecasts produced between 8<sup>th</sup> March to 29<sup>th</sup> November 2020 for 81 countries. We evaluated the robustness of the forecasts produced in real-time using relative error, coverage probability, and comparisons with null models. During the 39-week period covered by this study, both the short- and medium-term forecasts captured well the epidemic trajectory across different waves of COVID-19 infections with small relative errors over the forecast horizon. The model was well calibrated with 56.3% and 45.6% of the observations lying in the 50% Credible Interval in 1-week and 4-week ahead forecasts respectively. The retrospective evaluation of our models shows that simple transmission models calibrated using routine disease surveillance data can reliably capture the epidemic trajectory in multiple countries. The medium-term forecasts can be used in conjunction with the short-term forecasts of COVID-19 mortality as a useful planning tool as countries continue to relax public health measures. |
| spellingShingle | Bhatia, S Parag, KV Wardle, J Nash, RK Imai, N Van Elsland, SL Lassmann, B Brownstein, JS Desai, A Herringer, M Sewalk, K Loeb, SC Ramatowski, J Cuomo-Dannenburg, G Jauneikaite, E Unwin, HJT Riley, S Ferguson, N Donnelly, C Cori, A Nouvellet, P Retrospective evaluation of real-time estimates of global COVID-19 transmission trends and mortality forecasts |
| title | Retrospective evaluation of real-time estimates of global COVID-19 transmission trends and mortality forecasts |
| title_full | Retrospective evaluation of real-time estimates of global COVID-19 transmission trends and mortality forecasts |
| title_fullStr | Retrospective evaluation of real-time estimates of global COVID-19 transmission trends and mortality forecasts |
| title_full_unstemmed | Retrospective evaluation of real-time estimates of global COVID-19 transmission trends and mortality forecasts |
| title_short | Retrospective evaluation of real-time estimates of global COVID-19 transmission trends and mortality forecasts |
| title_sort | retrospective evaluation of real time estimates of global covid 19 transmission trends and mortality forecasts |
| work_keys_str_mv | AT bhatias retrospectiveevaluationofrealtimeestimatesofglobalcovid19transmissiontrendsandmortalityforecasts AT paragkv retrospectiveevaluationofrealtimeestimatesofglobalcovid19transmissiontrendsandmortalityforecasts AT wardlej retrospectiveevaluationofrealtimeestimatesofglobalcovid19transmissiontrendsandmortalityforecasts AT nashrk retrospectiveevaluationofrealtimeestimatesofglobalcovid19transmissiontrendsandmortalityforecasts AT imain retrospectiveevaluationofrealtimeestimatesofglobalcovid19transmissiontrendsandmortalityforecasts AT vanelslandsl retrospectiveevaluationofrealtimeestimatesofglobalcovid19transmissiontrendsandmortalityforecasts AT lassmannb retrospectiveevaluationofrealtimeestimatesofglobalcovid19transmissiontrendsandmortalityforecasts AT brownsteinjs retrospectiveevaluationofrealtimeestimatesofglobalcovid19transmissiontrendsandmortalityforecasts AT desaia retrospectiveevaluationofrealtimeestimatesofglobalcovid19transmissiontrendsandmortalityforecasts AT herringerm retrospectiveevaluationofrealtimeestimatesofglobalcovid19transmissiontrendsandmortalityforecasts AT sewalkk retrospectiveevaluationofrealtimeestimatesofglobalcovid19transmissiontrendsandmortalityforecasts AT loebsc retrospectiveevaluationofrealtimeestimatesofglobalcovid19transmissiontrendsandmortalityforecasts AT ramatowskij retrospectiveevaluationofrealtimeestimatesofglobalcovid19transmissiontrendsandmortalityforecasts AT cuomodannenburgg retrospectiveevaluationofrealtimeestimatesofglobalcovid19transmissiontrendsandmortalityforecasts AT jauneikaitee retrospectiveevaluationofrealtimeestimatesofglobalcovid19transmissiontrendsandmortalityforecasts AT unwinhjt retrospectiveevaluationofrealtimeestimatesofglobalcovid19transmissiontrendsandmortalityforecasts AT rileys retrospectiveevaluationofrealtimeestimatesofglobalcovid19transmissiontrendsandmortalityforecasts AT fergusonn retrospectiveevaluationofrealtimeestimatesofglobalcovid19transmissiontrendsandmortalityforecasts AT donnellyc retrospectiveevaluationofrealtimeestimatesofglobalcovid19transmissiontrendsandmortalityforecasts AT coria retrospectiveevaluationofrealtimeestimatesofglobalcovid19transmissiontrendsandmortalityforecasts AT nouvelletp retrospectiveevaluationofrealtimeestimatesofglobalcovid19transmissiontrendsandmortalityforecasts |