Serology Assays Used in SARS-CoV-2 Seroprevalence Surveys Worldwide: A Systematic Review and Meta-Analysis of Assay Features, Testing Algorithms, and Performance
Background: Many serological assays to detect SARS-CoV-2 antibodies were developed during the COVID-19 pandemic. Differences in the detection mechanism of SARS-CoV-2 serological assays limited the comparability of seroprevalence estimates for populations being tested. Methods: We conducted a systema...
| Main Authors: | , , , , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2022-11-01
|
| Series: | Vaccines |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2076-393X/10/12/2000 |
| _version_ | 1797455013934530560 |
|---|---|
| author | Xiaomeng Ma Zihan Li Mairead G. Whelan Dayoung Kim Christian Cao Mercedes Yanes-Lane Tingting Yan Thomas Jaenisch May Chu David A. Clifton Lorenzo Subissi Niklas Bobrovitz Rahul K. Arora |
| author_facet | Xiaomeng Ma Zihan Li Mairead G. Whelan Dayoung Kim Christian Cao Mercedes Yanes-Lane Tingting Yan Thomas Jaenisch May Chu David A. Clifton Lorenzo Subissi Niklas Bobrovitz Rahul K. Arora |
| author_sort | Xiaomeng Ma |
| collection | DOAJ |
| description | Background: Many serological assays to detect SARS-CoV-2 antibodies were developed during the COVID-19 pandemic. Differences in the detection mechanism of SARS-CoV-2 serological assays limited the comparability of seroprevalence estimates for populations being tested. Methods: We conducted a systematic review and meta-analysis of serological assays used in SARS-CoV-2 population seroprevalence surveys, searching for published articles, preprints, institutional sources, and grey literature between 1 January 2020, and 19 November 2021. We described features of all identified assays and mapped performance metrics by the manufacturers, third-party head-to-head, and independent group evaluations. We compared the reported assay performance by evaluation source with a mixed-effect beta regression model. A simulation was run to quantify how biased assay performance affects population seroprevalence estimates with test adjustment. Results: Among 1807 included serosurveys, 192 distinctive commercial assays and 380 self-developed assays were identified. According to manufacturers, 28.6% of all commercial assays met WHO criteria for emergency use (sensitivity [Sn.] >= 90.0%, specificity [Sp.] >= 97.0%). However, manufacturers overstated the absolute values of Sn. of commercial assays by 1.0% [0.1, 1.4%] and 3.3% [2.7, 3.4%], and Sp. by 0.9% [0.9, 0.9%] and 0.2% [−0.1, 0.4%] compared to third-party and independent evaluations, respectively. Reported performance data was not sufficient to support a similar analysis for self-developed assays. Simulations indicate that inaccurate Sn. and Sp. can bias seroprevalence estimates adjusted for assay performance; the error level changes with the background seroprevalence. Conclusions: The Sn. and Sp. of the serological assay are not fixed properties, but varying features depending on the testing population. To achieve precise population estimates and to ensure the comparability of seroprevalence, serosurveys should select assays with high performance validated not only by their manufacturers and adjust seroprevalence estimates based on assured performance data. More investigation should be directed to consolidating the performance of self-developed assays. |
| first_indexed | 2024-03-09T15:45:27Z |
| format | Article |
| id | doaj.art-1d02efe056024ddf8c6156681ecda1b8 |
| institution | Directory Open Access Journal |
| issn | 2076-393X |
| language | English |
| last_indexed | 2024-03-09T15:45:27Z |
| publishDate | 2022-11-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Vaccines |
| spelling | doaj.art-1d02efe056024ddf8c6156681ecda1b82023-11-24T18:31:01ZengMDPI AGVaccines2076-393X2022-11-011012200010.3390/vaccines10122000Serology Assays Used in SARS-CoV-2 Seroprevalence Surveys Worldwide: A Systematic Review and Meta-Analysis of Assay Features, Testing Algorithms, and PerformanceXiaomeng Ma0Zihan Li1Mairead G. Whelan2Dayoung Kim3Christian Cao4Mercedes Yanes-Lane5Tingting Yan6Thomas Jaenisch7May Chu8David A. Clifton9Lorenzo Subissi10Niklas Bobrovitz11Rahul K. Arora12Cumming School of Medicine, University of Calgary, Calgary, AB T2N 4N1, CanadaCumming School of Medicine, University of Calgary, Calgary, AB T2N 4N1, CanadaCumming School of Medicine, University of Calgary, Calgary, AB T2N 4N1, CanadaFaculty of Science, University of Calgary, Calgary, AB T2N 1N4, CanadaCumming School of Medicine, University of Calgary, Calgary, AB T2N 4N1, CanadaCOVID-19 Immunity Task Force, McGill University, Montreal, QC H3A 0G4, CanadaCumming School of Medicine, University of Calgary, Calgary, AB T2N 4N1, CanadaDepartment of Epidemiology & Center for Global Health, Colorado School of Public Health, Aurora, CO 80045, USADepartment of Epidemiology & Center for Global Health, Colorado School of Public Health, Aurora, CO 80045, USAInstitute of Biomedical Engineering, University of Oxford, Oxford OX3 7DQ, UKWorld Health Organization, 1211 Geneva, SwitzerlandTemerty Faculty of Medicine, University of Toronto, Toronto, ON M5S 1A8, CanadaCumming School of Medicine, University of Calgary, Calgary, AB T2N 4N1, CanadaBackground: Many serological assays to detect SARS-CoV-2 antibodies were developed during the COVID-19 pandemic. Differences in the detection mechanism of SARS-CoV-2 serological assays limited the comparability of seroprevalence estimates for populations being tested. Methods: We conducted a systematic review and meta-analysis of serological assays used in SARS-CoV-2 population seroprevalence surveys, searching for published articles, preprints, institutional sources, and grey literature between 1 January 2020, and 19 November 2021. We described features of all identified assays and mapped performance metrics by the manufacturers, third-party head-to-head, and independent group evaluations. We compared the reported assay performance by evaluation source with a mixed-effect beta regression model. A simulation was run to quantify how biased assay performance affects population seroprevalence estimates with test adjustment. Results: Among 1807 included serosurveys, 192 distinctive commercial assays and 380 self-developed assays were identified. According to manufacturers, 28.6% of all commercial assays met WHO criteria for emergency use (sensitivity [Sn.] >= 90.0%, specificity [Sp.] >= 97.0%). However, manufacturers overstated the absolute values of Sn. of commercial assays by 1.0% [0.1, 1.4%] and 3.3% [2.7, 3.4%], and Sp. by 0.9% [0.9, 0.9%] and 0.2% [−0.1, 0.4%] compared to third-party and independent evaluations, respectively. Reported performance data was not sufficient to support a similar analysis for self-developed assays. Simulations indicate that inaccurate Sn. and Sp. can bias seroprevalence estimates adjusted for assay performance; the error level changes with the background seroprevalence. Conclusions: The Sn. and Sp. of the serological assay are not fixed properties, but varying features depending on the testing population. To achieve precise population estimates and to ensure the comparability of seroprevalence, serosurveys should select assays with high performance validated not only by their manufacturers and adjust seroprevalence estimates based on assured performance data. More investigation should be directed to consolidating the performance of self-developed assays.https://www.mdpi.com/2076-393X/10/12/2000serological assayseroprevalenceperformancesensitivityspecificityevaluation |
| spellingShingle | Xiaomeng Ma Zihan Li Mairead G. Whelan Dayoung Kim Christian Cao Mercedes Yanes-Lane Tingting Yan Thomas Jaenisch May Chu David A. Clifton Lorenzo Subissi Niklas Bobrovitz Rahul K. Arora Serology Assays Used in SARS-CoV-2 Seroprevalence Surveys Worldwide: A Systematic Review and Meta-Analysis of Assay Features, Testing Algorithms, and Performance Vaccines serological assay seroprevalence performance sensitivity specificity evaluation |
| title | Serology Assays Used in SARS-CoV-2 Seroprevalence Surveys Worldwide: A Systematic Review and Meta-Analysis of Assay Features, Testing Algorithms, and Performance |
| title_full | Serology Assays Used in SARS-CoV-2 Seroprevalence Surveys Worldwide: A Systematic Review and Meta-Analysis of Assay Features, Testing Algorithms, and Performance |
| title_fullStr | Serology Assays Used in SARS-CoV-2 Seroprevalence Surveys Worldwide: A Systematic Review and Meta-Analysis of Assay Features, Testing Algorithms, and Performance |
| title_full_unstemmed | Serology Assays Used in SARS-CoV-2 Seroprevalence Surveys Worldwide: A Systematic Review and Meta-Analysis of Assay Features, Testing Algorithms, and Performance |
| title_short | Serology Assays Used in SARS-CoV-2 Seroprevalence Surveys Worldwide: A Systematic Review and Meta-Analysis of Assay Features, Testing Algorithms, and Performance |
| title_sort | serology assays used in sars cov 2 seroprevalence surveys worldwide a systematic review and meta analysis of assay features testing algorithms and performance |
| topic | serological assay seroprevalence performance sensitivity specificity evaluation |
| url | https://www.mdpi.com/2076-393X/10/12/2000 |
| work_keys_str_mv | AT xiaomengma serologyassaysusedinsarscov2seroprevalencesurveysworldwideasystematicreviewandmetaanalysisofassayfeaturestestingalgorithmsandperformance AT zihanli serologyassaysusedinsarscov2seroprevalencesurveysworldwideasystematicreviewandmetaanalysisofassayfeaturestestingalgorithmsandperformance AT maireadgwhelan serologyassaysusedinsarscov2seroprevalencesurveysworldwideasystematicreviewandmetaanalysisofassayfeaturestestingalgorithmsandperformance AT dayoungkim serologyassaysusedinsarscov2seroprevalencesurveysworldwideasystematicreviewandmetaanalysisofassayfeaturestestingalgorithmsandperformance AT christiancao serologyassaysusedinsarscov2seroprevalencesurveysworldwideasystematicreviewandmetaanalysisofassayfeaturestestingalgorithmsandperformance AT mercedesyaneslane serologyassaysusedinsarscov2seroprevalencesurveysworldwideasystematicreviewandmetaanalysisofassayfeaturestestingalgorithmsandperformance AT tingtingyan serologyassaysusedinsarscov2seroprevalencesurveysworldwideasystematicreviewandmetaanalysisofassayfeaturestestingalgorithmsandperformance AT thomasjaenisch serologyassaysusedinsarscov2seroprevalencesurveysworldwideasystematicreviewandmetaanalysisofassayfeaturestestingalgorithmsandperformance AT maychu serologyassaysusedinsarscov2seroprevalencesurveysworldwideasystematicreviewandmetaanalysisofassayfeaturestestingalgorithmsandperformance AT davidaclifton serologyassaysusedinsarscov2seroprevalencesurveysworldwideasystematicreviewandmetaanalysisofassayfeaturestestingalgorithmsandperformance AT lorenzosubissi serologyassaysusedinsarscov2seroprevalencesurveysworldwideasystematicreviewandmetaanalysisofassayfeaturestestingalgorithmsandperformance AT niklasbobrovitz serologyassaysusedinsarscov2seroprevalencesurveysworldwideasystematicreviewandmetaanalysisofassayfeaturestestingalgorithmsandperformance AT rahulkarora serologyassaysusedinsarscov2seroprevalencesurveysworldwideasystematicreviewandmetaanalysisofassayfeaturestestingalgorithmsandperformance |