Accelerating Bayesian inference in computationally expensive computer models using local and global approximations
Thesis: Ph. D., Massachusetts Institute of Technology, Department of Aeronautics and Astronautics, 2014.
| Main Author: | |
|---|---|
| Other Authors: | |
| Format: | Thesis |
| Language: | eng |
| Published: |
Massachusetts Institute of Technology
2014
|
| Subjects: | |
| Online Access: | http://hdl.handle.net/1721.1/90599 |
| _version_ | 1811082264767889408 |
|---|---|
| author | Conrad, Patrick Raymond |
| author2 | Youssef M. Marzouk. |
| author_facet | Youssef M. Marzouk. Conrad, Patrick Raymond |
| author_sort | Conrad, Patrick Raymond |
| collection | MIT |
| description | Thesis: Ph. D., Massachusetts Institute of Technology, Department of Aeronautics and Astronautics, 2014. |
| first_indexed | 2024-09-23T12:00:21Z |
| format | Thesis |
| id | mit-1721.1/90599 |
| institution | Massachusetts Institute of Technology |
| language | eng |
| last_indexed | 2024-09-23T12:00:21Z |
| publishDate | 2014 |
| publisher | Massachusetts Institute of Technology |
| record_format | dspace |
| spelling | mit-1721.1/905992019-04-12T20:48:10Z Accelerating Bayesian inference in computationally expensive computer models using local and global approximations Conrad, Patrick Raymond Youssef M. Marzouk. Massachusetts Institute of Technology. Department of Aeronautics and Astronautics. Massachusetts Institute of Technology. Department of Aeronautics and Astronautics. Aeronautics and Astronautics. Thesis: Ph. D., Massachusetts Institute of Technology, Department of Aeronautics and Astronautics, 2014. This electronic version was submitted by the student author. The certified thesis is available in the Institute Archives and Special Collections. Cataloged from student-submitted PDF version of thesis. Includes bibliographical references (pages 129-138). Computational models of complex phenomena are an important resource for scientists and engineers. However, many state-of-the-art simulations of physical systems are computationally expensive to evaluate and are black box-meaning that they can be run, but their internal workings cannot be inspected or changed. Directly applying uncertainty quantification algorithms, such as those for forward uncertainty propagation or Bayesian inference, to these types of models is often intractable because the analyses use many evaluations of the model. Fortunately, many physical systems are well behaved, in the sense that they may be efficiently approximated with a modest number of carefully chosen samples. This thesis develops global and local approximation strategies that can be applied to black-box models to reduce the cost of forward uncertainty quantification and Bayesian inference. First, we develop an efficient strategy for constructing global approximations using an orthonormal polynomial basis. We rigorously construct a Smolyak pseudospectral algorithm, which uses sparse sample sets to efficiently extract information from loosely coupled functions. We provide a theoretical discussion of the behavior and accuracy of this algorithm, concluding that it has favorable convergence characteristics. We make this strategy efficient in practice by introducing a greedy heuristic that adaptively identifies and explores the important input dimensions, or combinations thereof. When the approximation is used within Bayesian inference, however, it is difficult to translate the theoretical behavior of the global approximations into practical controls on the error induced in the resulting posterior distribution. Thus, the second part of this thesis introduces a new framework for accelerating MCMC algorithms by constructing local surrogates of the computational model within the Metropolis-Hastings kernel, borrowing ideas from deterministic approximation theory, optimization, and experimental design. Exploiting useful convergence characteristics of local approximations, we prove the ergodicity of our approximate Markov chain and show that it samples asymptotically from the exact posterior distribution of interest. Our theoretical results reinforce the key observation underlying this work: when the likelihood has some local regularity, the number of model evaluations per MCMC step can be greatly reduced, without incurring significant bias in the Monte Carlo average. We illustrate that the inference framework is robust and extensible by describing variations that use different approximation families, MCMC kernels, and computational environments. Our numerical experiments demonstrate order-of-magnitude reductions in the number of forward model evaluations used in representative ODE or PDE inference problems, in both real and synthetic data examples. Finally, we demonstrate the local approximation algorithm by performing parameter inference for the ice-ocean coupling in Pine Island Glacier, Antarctica. This problem constitutes a challenging domain for inference and an important application in climate science. We perform simulated inference, comparing synthetic data to predictions from the MIT General Circulation Model, a state-of-the-art ocean simulation. The results reveal some information about parameter sensitivity, but we ultimately conclude that richer data is necessary to constrain the model parameters. In this example, applying our approximation techniques reduced the cost of the inference by a factor of five to ten, taking weeks instead of months, providing evidence that our techniques can make Bayesian inference on large-scale computational models more tractable. by Patrick Raymond Conrad. Ph. D. 2014-10-07T19:17:21Z 2014-10-07T19:17:21Z 2014 2014 Thesis http://hdl.handle.net/1721.1/90599 890383899 eng M.I.T. theses are protected by copyright. They may be viewed from this source for any purpose, but reproduction or distribution in any format is prohibited without written permission. See provided URL for inquiries about permission. http://dspace.mit.edu/handle/1721.1/7582 138 pages application/pdf Massachusetts Institute of Technology |
| spellingShingle | Aeronautics and Astronautics. Conrad, Patrick Raymond Accelerating Bayesian inference in computationally expensive computer models using local and global approximations |
| title | Accelerating Bayesian inference in computationally expensive computer models using local and global approximations |
| title_full | Accelerating Bayesian inference in computationally expensive computer models using local and global approximations |
| title_fullStr | Accelerating Bayesian inference in computationally expensive computer models using local and global approximations |
| title_full_unstemmed | Accelerating Bayesian inference in computationally expensive computer models using local and global approximations |
| title_short | Accelerating Bayesian inference in computationally expensive computer models using local and global approximations |
| title_sort | accelerating bayesian inference in computationally expensive computer models using local and global approximations |
| topic | Aeronautics and Astronautics. |
| url | http://hdl.handle.net/1721.1/90599 |
| work_keys_str_mv | AT conradpatrickraymond acceleratingbayesianinferenceincomputationallyexpensivecomputermodelsusinglocalandglobalapproximations |