Characterization and Valuation of the Uncertainty of Calibrated Parameters in Microsimulation Decision Models
Background: We evaluated the implications of different approaches to characterize the uncertainty of calibrated parameters of microsimulation decision models (DMs) and quantified the value of such uncertainty in decision making.Methods: We calibrated the natural history model of CRC to simulated epi...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Frontiers Media S.A.
2022-05-01
|
| Series: | Frontiers in Physiology |
| Subjects: | |
| Online Access: | https://www.frontiersin.org/articles/10.3389/fphys.2022.780917/full |
| _version_ | 1818269614458535936 |
|---|---|
| author | Fernando Alarid-Escudero Amy B. Knudsen Jonathan Ozik Jonathan Ozik Nicholson Collier Nicholson Collier Karen M. Kuntz |
| author_facet | Fernando Alarid-Escudero Amy B. Knudsen Jonathan Ozik Jonathan Ozik Nicholson Collier Nicholson Collier Karen M. Kuntz |
| author_sort | Fernando Alarid-Escudero |
| collection | DOAJ |
| description | Background: We evaluated the implications of different approaches to characterize the uncertainty of calibrated parameters of microsimulation decision models (DMs) and quantified the value of such uncertainty in decision making.Methods: We calibrated the natural history model of CRC to simulated epidemiological data with different degrees of uncertainty and obtained the joint posterior distribution of the parameters using a Bayesian approach. We conducted a probabilistic sensitivity analysis (PSA) on all the model parameters with different characterizations of the uncertainty of the calibrated parameters. We estimated the value of uncertainty of the various characterizations with a value of information analysis. We conducted all analyses using high-performance computing resources running the Extreme-scale Model Exploration with Swift (EMEWS) framework.Results: The posterior distribution had a high correlation among some parameters. The parameters of the Weibull hazard function for the age of onset of adenomas had the highest posterior correlation of −0.958. When comparing full posterior distributions and the maximum-a-posteriori estimate of the calibrated parameters, there is little difference in the spread of the distribution of the CEA outcomes with a similar expected value of perfect information (EVPI) of $653 and $685, respectively, at a willingness-to-pay (WTP) threshold of $66,000 per quality-adjusted life year (QALY). Ignoring correlation on the calibrated parameters’ posterior distribution produced the broadest distribution of CEA outcomes and the highest EVPI of $809 at the same WTP threshold.Conclusion: Different characterizations of the uncertainty of calibrated parameters affect the expected value of eliminating parametric uncertainty on the CEA. Ignoring inherent correlation among calibrated parameters on a PSA overestimates the value of uncertainty. |
| first_indexed | 2024-12-12T20:57:11Z |
| format | Article |
| id | doaj.art-bb8a545b8fff4a16b9f179b9524a41ee |
| institution | Directory Open Access Journal |
| issn | 1664-042X |
| language | English |
| last_indexed | 2024-12-12T20:57:11Z |
| publishDate | 2022-05-01 |
| publisher | Frontiers Media S.A. |
| record_format | Article |
| series | Frontiers in Physiology |
| spelling | doaj.art-bb8a545b8fff4a16b9f179b9524a41ee2022-12-22T00:12:16ZengFrontiers Media S.A.Frontiers in Physiology1664-042X2022-05-011310.3389/fphys.2022.780917780917Characterization and Valuation of the Uncertainty of Calibrated Parameters in Microsimulation Decision ModelsFernando Alarid-Escudero0Amy B. Knudsen1Jonathan Ozik2Jonathan Ozik3Nicholson Collier4Nicholson Collier5Karen M. Kuntz6Division of Public Administration, Center for Research and Teaching in Economics (CIDE), Aguascalientes, MexicoInstitute for Technology Assessment, Massachusetts General Hospital, Boston, MA, United StatesDecision and Infrastructure Sciences Division, Argonne National Laboratory, Argonne, IL, United StatesConsortium for Advanced Science and Engineering, University of Chicago, Chicago, IL, United StatesDecision and Infrastructure Sciences Division, Argonne National Laboratory, Argonne, IL, United StatesConsortium for Advanced Science and Engineering, University of Chicago, Chicago, IL, United StatesDivision of Health Policy and Management, University of Minnesota School of Public Health, Minneapolis, MN, United StatesBackground: We evaluated the implications of different approaches to characterize the uncertainty of calibrated parameters of microsimulation decision models (DMs) and quantified the value of such uncertainty in decision making.Methods: We calibrated the natural history model of CRC to simulated epidemiological data with different degrees of uncertainty and obtained the joint posterior distribution of the parameters using a Bayesian approach. We conducted a probabilistic sensitivity analysis (PSA) on all the model parameters with different characterizations of the uncertainty of the calibrated parameters. We estimated the value of uncertainty of the various characterizations with a value of information analysis. We conducted all analyses using high-performance computing resources running the Extreme-scale Model Exploration with Swift (EMEWS) framework.Results: The posterior distribution had a high correlation among some parameters. The parameters of the Weibull hazard function for the age of onset of adenomas had the highest posterior correlation of −0.958. When comparing full posterior distributions and the maximum-a-posteriori estimate of the calibrated parameters, there is little difference in the spread of the distribution of the CEA outcomes with a similar expected value of perfect information (EVPI) of $653 and $685, respectively, at a willingness-to-pay (WTP) threshold of $66,000 per quality-adjusted life year (QALY). Ignoring correlation on the calibrated parameters’ posterior distribution produced the broadest distribution of CEA outcomes and the highest EVPI of $809 at the same WTP threshold.Conclusion: Different characterizations of the uncertainty of calibrated parameters affect the expected value of eliminating parametric uncertainty on the CEA. Ignoring inherent correlation among calibrated parameters on a PSA overestimates the value of uncertainty.https://www.frontiersin.org/articles/10.3389/fphys.2022.780917/fullmicrosimulation modelsuncertainty quantificationcalibrationBayesianvalue of information analysisdecision-analytic models |
| spellingShingle | Fernando Alarid-Escudero Amy B. Knudsen Jonathan Ozik Jonathan Ozik Nicholson Collier Nicholson Collier Karen M. Kuntz Characterization and Valuation of the Uncertainty of Calibrated Parameters in Microsimulation Decision Models Frontiers in Physiology microsimulation models uncertainty quantification calibration Bayesian value of information analysis decision-analytic models |
| title | Characterization and Valuation of the Uncertainty of Calibrated Parameters in Microsimulation Decision Models |
| title_full | Characterization and Valuation of the Uncertainty of Calibrated Parameters in Microsimulation Decision Models |
| title_fullStr | Characterization and Valuation of the Uncertainty of Calibrated Parameters in Microsimulation Decision Models |
| title_full_unstemmed | Characterization and Valuation of the Uncertainty of Calibrated Parameters in Microsimulation Decision Models |
| title_short | Characterization and Valuation of the Uncertainty of Calibrated Parameters in Microsimulation Decision Models |
| title_sort | characterization and valuation of the uncertainty of calibrated parameters in microsimulation decision models |
| topic | microsimulation models uncertainty quantification calibration Bayesian value of information analysis decision-analytic models |
| url | https://www.frontiersin.org/articles/10.3389/fphys.2022.780917/full |
| work_keys_str_mv | AT fernandoalaridescudero characterizationandvaluationoftheuncertaintyofcalibratedparametersinmicrosimulationdecisionmodels AT amybknudsen characterizationandvaluationoftheuncertaintyofcalibratedparametersinmicrosimulationdecisionmodels AT jonathanozik characterizationandvaluationoftheuncertaintyofcalibratedparametersinmicrosimulationdecisionmodels AT jonathanozik characterizationandvaluationoftheuncertaintyofcalibratedparametersinmicrosimulationdecisionmodels AT nicholsoncollier characterizationandvaluationoftheuncertaintyofcalibratedparametersinmicrosimulationdecisionmodels AT nicholsoncollier characterizationandvaluationoftheuncertaintyofcalibratedparametersinmicrosimulationdecisionmodels AT karenmkuntz characterizationandvaluationoftheuncertaintyofcalibratedparametersinmicrosimulationdecisionmodels |