Developing and Validating Metamodels of a Microsimulation Model of Infant HIV Testing and Screening Strategies Used in a Decision Support Tool for Health Policy Makers
Background. Metamodels can simplify complex health policy models and yield instantaneous results to inform policy decisions. We investigated the predictive validity of linear regression metamodels used to support a real-time decision-making tool that compares infant HIV testing/screening strategies....
| Main Authors: | , , , , , , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
SAGE Publishing
2020-06-01
|
| Series: | MDM Policy & Practice |
| Online Access: | https://doi.org/10.1177/2381468320932894 |
| _version_ | 1818289314041167872 |
|---|---|
| author | Djøra I. Soeteman Stephen C. Resch Hawre Jalal Caitlin M. Dugdale Martina Penazzato Milton C. Weinstein Andrew Phillips Taige Hou Elaine J. Abrams Lorna Dunning Marie-Louise Newell Pamela P. Pei Kenneth A. Freedberg Rochelle P. Walensky Andrea L. Ciaranello |
| author_facet | Djøra I. Soeteman Stephen C. Resch Hawre Jalal Caitlin M. Dugdale Martina Penazzato Milton C. Weinstein Andrew Phillips Taige Hou Elaine J. Abrams Lorna Dunning Marie-Louise Newell Pamela P. Pei Kenneth A. Freedberg Rochelle P. Walensky Andrea L. Ciaranello |
| author_sort | Djøra I. Soeteman |
| collection | DOAJ |
| description | Background. Metamodels can simplify complex health policy models and yield instantaneous results to inform policy decisions. We investigated the predictive validity of linear regression metamodels used to support a real-time decision-making tool that compares infant HIV testing/screening strategies. Methods. We developed linear regression metamodels of the Cost-Effectiveness of Preventing AIDS Complications Pediatric (CEPAC-P) microsimulation model used to predict life expectancy and lifetime HIV-related costs/person of two infant HIV testing/screening programs in South Africa. Metamodel performance was assessed with cross-validation and Bland-Altman plots, showing between-method differences in predicted outcomes against their means. Predictive validity was determined by the percentage of simulations in which the metamodels accurately predicted the strategy with the greatest net health benefit (NHB) as projected by the CEPAC-P model. We introduced a zone of indifference and investigated the width needed to produce between-method agreement in 95% of the simulations. We also calculated NHB losses from “wrong” decisions by the metamodel. Results. In cross-validation, linear regression metamodels accurately approximated CEPAC-P-projected outcomes. For life expectancy, Bland-Altman plots showed good agreement between CEPAC-P and the metamodel (within 1.1 life-months difference). For costs, 95% of between-method differences were within $65/person. The metamodels predicted the same optimal strategy as the CEPAC-P model in 87.7% of simulations, increasing to 95% with a zone of indifference of 0.24 life-months ( ∼ 7 days). The losses in health benefits due to “wrong” choices by the metamodel were modest (range: 0.0002–1.1 life-months). Conclusions. For this policy question, linear regression metamodels offered sufficient predictive validity for the optimal testing strategy as compared with the CEPAC-P model. Metamodels can simulate different scenarios in real time, based on sets of input parameters that can be depicted in a widely accessible decision-support tool. |
| first_indexed | 2024-12-13T02:10:18Z |
| format | Article |
| id | doaj.art-553a5ceff5124f1281ac37ebe18e7af0 |
| institution | Directory Open Access Journal |
| issn | 2381-4683 |
| language | English |
| last_indexed | 2024-12-13T02:10:18Z |
| publishDate | 2020-06-01 |
| publisher | SAGE Publishing |
| record_format | Article |
| series | MDM Policy & Practice |
| spelling | doaj.art-553a5ceff5124f1281ac37ebe18e7af02022-12-22T00:03:02ZengSAGE PublishingMDM Policy & Practice2381-46832020-06-01510.1177/2381468320932894Developing and Validating Metamodels of a Microsimulation Model of Infant HIV Testing and Screening Strategies Used in a Decision Support Tool for Health Policy MakersDjøra I. SoetemanStephen C. ReschHawre JalalCaitlin M. DugdaleMartina PenazzatoMilton C. WeinsteinAndrew PhillipsTaige HouElaine J. AbramsLorna DunningMarie-Louise NewellPamela P. PeiKenneth A. FreedbergRochelle P. WalenskyAndrea L. CiaranelloBackground. Metamodels can simplify complex health policy models and yield instantaneous results to inform policy decisions. We investigated the predictive validity of linear regression metamodels used to support a real-time decision-making tool that compares infant HIV testing/screening strategies. Methods. We developed linear regression metamodels of the Cost-Effectiveness of Preventing AIDS Complications Pediatric (CEPAC-P) microsimulation model used to predict life expectancy and lifetime HIV-related costs/person of two infant HIV testing/screening programs in South Africa. Metamodel performance was assessed with cross-validation and Bland-Altman plots, showing between-method differences in predicted outcomes against their means. Predictive validity was determined by the percentage of simulations in which the metamodels accurately predicted the strategy with the greatest net health benefit (NHB) as projected by the CEPAC-P model. We introduced a zone of indifference and investigated the width needed to produce between-method agreement in 95% of the simulations. We also calculated NHB losses from “wrong” decisions by the metamodel. Results. In cross-validation, linear regression metamodels accurately approximated CEPAC-P-projected outcomes. For life expectancy, Bland-Altman plots showed good agreement between CEPAC-P and the metamodel (within 1.1 life-months difference). For costs, 95% of between-method differences were within $65/person. The metamodels predicted the same optimal strategy as the CEPAC-P model in 87.7% of simulations, increasing to 95% with a zone of indifference of 0.24 life-months ( ∼ 7 days). The losses in health benefits due to “wrong” choices by the metamodel were modest (range: 0.0002–1.1 life-months). Conclusions. For this policy question, linear regression metamodels offered sufficient predictive validity for the optimal testing strategy as compared with the CEPAC-P model. Metamodels can simulate different scenarios in real time, based on sets of input parameters that can be depicted in a widely accessible decision-support tool.https://doi.org/10.1177/2381468320932894 |
| spellingShingle | Djøra I. Soeteman Stephen C. Resch Hawre Jalal Caitlin M. Dugdale Martina Penazzato Milton C. Weinstein Andrew Phillips Taige Hou Elaine J. Abrams Lorna Dunning Marie-Louise Newell Pamela P. Pei Kenneth A. Freedberg Rochelle P. Walensky Andrea L. Ciaranello Developing and Validating Metamodels of a Microsimulation Model of Infant HIV Testing and Screening Strategies Used in a Decision Support Tool for Health Policy Makers MDM Policy & Practice |
| title | Developing and Validating Metamodels of a Microsimulation Model of Infant HIV Testing and Screening Strategies Used in a Decision Support Tool for Health Policy Makers |
| title_full | Developing and Validating Metamodels of a Microsimulation Model of Infant HIV Testing and Screening Strategies Used in a Decision Support Tool for Health Policy Makers |
| title_fullStr | Developing and Validating Metamodels of a Microsimulation Model of Infant HIV Testing and Screening Strategies Used in a Decision Support Tool for Health Policy Makers |
| title_full_unstemmed | Developing and Validating Metamodels of a Microsimulation Model of Infant HIV Testing and Screening Strategies Used in a Decision Support Tool for Health Policy Makers |
| title_short | Developing and Validating Metamodels of a Microsimulation Model of Infant HIV Testing and Screening Strategies Used in a Decision Support Tool for Health Policy Makers |
| title_sort | developing and validating metamodels of a microsimulation model of infant hiv testing and screening strategies used in a decision support tool for health policy makers |
| url | https://doi.org/10.1177/2381468320932894 |
| work_keys_str_mv | AT djøraisoeteman developingandvalidatingmetamodelsofamicrosimulationmodelofinfanthivtestingandscreeningstrategiesusedinadecisionsupporttoolforhealthpolicymakers AT stephencresch developingandvalidatingmetamodelsofamicrosimulationmodelofinfanthivtestingandscreeningstrategiesusedinadecisionsupporttoolforhealthpolicymakers AT hawrejalal developingandvalidatingmetamodelsofamicrosimulationmodelofinfanthivtestingandscreeningstrategiesusedinadecisionsupporttoolforhealthpolicymakers AT caitlinmdugdale developingandvalidatingmetamodelsofamicrosimulationmodelofinfanthivtestingandscreeningstrategiesusedinadecisionsupporttoolforhealthpolicymakers AT martinapenazzato developingandvalidatingmetamodelsofamicrosimulationmodelofinfanthivtestingandscreeningstrategiesusedinadecisionsupporttoolforhealthpolicymakers AT miltoncweinstein developingandvalidatingmetamodelsofamicrosimulationmodelofinfanthivtestingandscreeningstrategiesusedinadecisionsupporttoolforhealthpolicymakers AT andrewphillips developingandvalidatingmetamodelsofamicrosimulationmodelofinfanthivtestingandscreeningstrategiesusedinadecisionsupporttoolforhealthpolicymakers AT taigehou developingandvalidatingmetamodelsofamicrosimulationmodelofinfanthivtestingandscreeningstrategiesusedinadecisionsupporttoolforhealthpolicymakers AT elainejabrams developingandvalidatingmetamodelsofamicrosimulationmodelofinfanthivtestingandscreeningstrategiesusedinadecisionsupporttoolforhealthpolicymakers AT lornadunning developingandvalidatingmetamodelsofamicrosimulationmodelofinfanthivtestingandscreeningstrategiesusedinadecisionsupporttoolforhealthpolicymakers AT marielouisenewell developingandvalidatingmetamodelsofamicrosimulationmodelofinfanthivtestingandscreeningstrategiesusedinadecisionsupporttoolforhealthpolicymakers AT pamelappei developingandvalidatingmetamodelsofamicrosimulationmodelofinfanthivtestingandscreeningstrategiesusedinadecisionsupporttoolforhealthpolicymakers AT kennethafreedberg developingandvalidatingmetamodelsofamicrosimulationmodelofinfanthivtestingandscreeningstrategiesusedinadecisionsupporttoolforhealthpolicymakers AT rochellepwalensky developingandvalidatingmetamodelsofamicrosimulationmodelofinfanthivtestingandscreeningstrategiesusedinadecisionsupporttoolforhealthpolicymakers AT andrealciaranello developingandvalidatingmetamodelsofamicrosimulationmodelofinfanthivtestingandscreeningstrategiesusedinadecisionsupporttoolforhealthpolicymakers |