Replicate Testing of Clinical Endpoints Can Prevent No-Go Decisions for Beneficial Vaccines
In vaccine efficacy trials, inaccurate counting of infection cases leads to systematic under-estimation—or “dilution”—of vaccine efficacy. In particular, if a sufficient fraction of observed cases are false positives, apparent efficacy will be greatly reduced, leading to unwarranted no-go decisions...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-09-01
|
| Series: | Vaccines |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2076-393X/11/9/1501 |
| _version_ | 1797576493680820224 |
|---|---|
| author | Daniel I. S. Rosenbloom Julie Dudášová Casey Davis Radha A. Railkar Nitin Mehrotra Jeffrey R. Sachs |
| author_facet | Daniel I. S. Rosenbloom Julie Dudášová Casey Davis Radha A. Railkar Nitin Mehrotra Jeffrey R. Sachs |
| author_sort | Daniel I. S. Rosenbloom |
| collection | DOAJ |
| description | In vaccine efficacy trials, inaccurate counting of infection cases leads to systematic under-estimation—or “dilution”—of vaccine efficacy. In particular, if a sufficient fraction of observed cases are false positives, apparent efficacy will be greatly reduced, leading to unwarranted no-go decisions in vaccine development. Here, we propose a range of replicate testing strategies to address this problem, considering the additional challenge of uncertainty in both infection incidence and diagnostic assay specificity/sensitivity. A strategy that counts an infection case only if a majority of replicate assays return a positive result can substantially reduce efficacy dilution for assays with non-systematic (i.e., “random”) errors. We also find that a cost-effective variant of this strategy, using confirmatory assays only if an initial assay is positive, yields a comparable benefit. In clinical trials, where frequent longitudinal samples are needed to detect short-lived infections, this “confirmatory majority rule” strategy can prevent the accumulation of false positives from magnifying efficacy dilution. When widespread public health screening is used for viruses, such as SARS-CoV-2, that have non-differentiating features or may be asymptomatic, these strategies can also serve to reduce unneeded isolations caused by false positives. |
| first_indexed | 2024-03-10T21:52:43Z |
| format | Article |
| id | doaj.art-9338473b398c4b5fa038fff778f3ca24 |
| institution | Directory Open Access Journal |
| issn | 2076-393X |
| language | English |
| last_indexed | 2024-03-10T21:52:43Z |
| publishDate | 2023-09-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Vaccines |
| spelling | doaj.art-9338473b398c4b5fa038fff778f3ca242023-11-19T13:19:51ZengMDPI AGVaccines2076-393X2023-09-01119150110.3390/vaccines11091501Replicate Testing of Clinical Endpoints Can Prevent No-Go Decisions for Beneficial VaccinesDaniel I. S. Rosenbloom0Julie Dudášová1Casey Davis2Radha A. Railkar3Nitin Mehrotra4Jeffrey R. Sachs5Quantitative Pharmacology and Pharmacometrics, Merck & Co., Inc., Rahway, NJ 07065, USAQuantitative Pharmacology and Pharmacometrics, MSD Czech Republic, 15000 Prague, Czech RepublicQuantitative Pharmacology and Pharmacometrics, Merck & Co., Inc., Rahway, NJ 07065, USABiostatistics and Research Decision Sciences, Merck & Co., Inc., Rahway, NJ 07065, USAQuantitative Pharmacology and Pharmacometrics, Merck & Co., Inc., Rahway, NJ 07065, USAQuantitative Pharmacology and Pharmacometrics, Merck & Co., Inc., Rahway, NJ 07065, USAIn vaccine efficacy trials, inaccurate counting of infection cases leads to systematic under-estimation—or “dilution”—of vaccine efficacy. In particular, if a sufficient fraction of observed cases are false positives, apparent efficacy will be greatly reduced, leading to unwarranted no-go decisions in vaccine development. Here, we propose a range of replicate testing strategies to address this problem, considering the additional challenge of uncertainty in both infection incidence and diagnostic assay specificity/sensitivity. A strategy that counts an infection case only if a majority of replicate assays return a positive result can substantially reduce efficacy dilution for assays with non-systematic (i.e., “random”) errors. We also find that a cost-effective variant of this strategy, using confirmatory assays only if an initial assay is positive, yields a comparable benefit. In clinical trials, where frequent longitudinal samples are needed to detect short-lived infections, this “confirmatory majority rule” strategy can prevent the accumulation of false positives from magnifying efficacy dilution. When widespread public health screening is used for viruses, such as SARS-CoV-2, that have non-differentiating features or may be asymptomatic, these strategies can also serve to reduce unneeded isolations caused by false positives.https://www.mdpi.com/2076-393X/11/9/1501clinical trial designvaccine efficacydiagnostic assayscase-countingfalse-positive ratediagnostic error |
| spellingShingle | Daniel I. S. Rosenbloom Julie Dudášová Casey Davis Radha A. Railkar Nitin Mehrotra Jeffrey R. Sachs Replicate Testing of Clinical Endpoints Can Prevent No-Go Decisions for Beneficial Vaccines Vaccines clinical trial design vaccine efficacy diagnostic assays case-counting false-positive rate diagnostic error |
| title | Replicate Testing of Clinical Endpoints Can Prevent No-Go Decisions for Beneficial Vaccines |
| title_full | Replicate Testing of Clinical Endpoints Can Prevent No-Go Decisions for Beneficial Vaccines |
| title_fullStr | Replicate Testing of Clinical Endpoints Can Prevent No-Go Decisions for Beneficial Vaccines |
| title_full_unstemmed | Replicate Testing of Clinical Endpoints Can Prevent No-Go Decisions for Beneficial Vaccines |
| title_short | Replicate Testing of Clinical Endpoints Can Prevent No-Go Decisions for Beneficial Vaccines |
| title_sort | replicate testing of clinical endpoints can prevent no go decisions for beneficial vaccines |
| topic | clinical trial design vaccine efficacy diagnostic assays case-counting false-positive rate diagnostic error |
| url | https://www.mdpi.com/2076-393X/11/9/1501 |
| work_keys_str_mv | AT danielisrosenbloom replicatetestingofclinicalendpointscanpreventnogodecisionsforbeneficialvaccines AT juliedudasova replicatetestingofclinicalendpointscanpreventnogodecisionsforbeneficialvaccines AT caseydavis replicatetestingofclinicalendpointscanpreventnogodecisionsforbeneficialvaccines AT radhaarailkar replicatetestingofclinicalendpointscanpreventnogodecisionsforbeneficialvaccines AT nitinmehrotra replicatetestingofclinicalendpointscanpreventnogodecisionsforbeneficialvaccines AT jeffreyrsachs replicatetestingofclinicalendpointscanpreventnogodecisionsforbeneficialvaccines |