MALA-within-Gibbs Samplers for High-Dimensional Distributions with Sparse Conditional Structure
© 2020 Society for Industrial and Applied Mathematics. Markov chain Monte Carlo (MCMC) samplers are numerical methods for drawing samples from a given target probability distribution. We discuss one particular MCMC sampler, the MALA-within-Gibbs sampler, from the theoretical and practical perspectiv...
| Main Authors: | , , |
|---|---|
| Other Authors: | |
| Format: | Article |
| Language: | English |
| Published: |
Society for Industrial & Applied Mathematics (SIAM)
2021
|
| Online Access: | https://hdl.handle.net/1721.1/135471 |
| _version_ | 1826188868183916544 |
|---|---|
| author | Tong, XT Morzfeld, M Marzouk, YM |
| author2 | Massachusetts Institute of Technology. Department of Aeronautics and Astronautics |
| author_facet | Massachusetts Institute of Technology. Department of Aeronautics and Astronautics Tong, XT Morzfeld, M Marzouk, YM |
| author_sort | Tong, XT |
| collection | MIT |
| description | © 2020 Society for Industrial and Applied Mathematics. Markov chain Monte Carlo (MCMC) samplers are numerical methods for drawing samples from a given target probability distribution. We discuss one particular MCMC sampler, the MALA-within-Gibbs sampler, from the theoretical and practical perspectives. We first show that the acceptance ratio and step size of this sampler are independent of the overall problem dimension when (i) the target distribution has sparse conditional structure, and (ii) this structure is reflected in the partial updating strategy of MALA-within-Gibbs. If, in addition, the target density is blockwise log-concave, then the sampler's convergence rate is independent of dimension. From a practical perspective, we expect that MALA-within-Gibbs is useful for solving high-dimensional Bayesian inference problems where the posterior exhibits sparse conditional structure at least approximately. In this context, a partitioning of the state that correctly reflects the sparse conditional structure must be found, and we illustrate this process in two numerical examples. We also discuss trade-offs between the block size used for partial updating and computational requirements that may increase with the number of blocks. |
| first_indexed | 2024-09-23T08:05:54Z |
| format | Article |
| id | mit-1721.1/135471 |
| institution | Massachusetts Institute of Technology |
| language | English |
| last_indexed | 2024-09-23T08:05:54Z |
| publishDate | 2021 |
| publisher | Society for Industrial & Applied Mathematics (SIAM) |
| record_format | dspace |
| spelling | mit-1721.1/1354712023-09-06T19:52:03Z MALA-within-Gibbs Samplers for High-Dimensional Distributions with Sparse Conditional Structure Tong, XT Morzfeld, M Marzouk, YM Massachusetts Institute of Technology. Department of Aeronautics and Astronautics © 2020 Society for Industrial and Applied Mathematics. Markov chain Monte Carlo (MCMC) samplers are numerical methods for drawing samples from a given target probability distribution. We discuss one particular MCMC sampler, the MALA-within-Gibbs sampler, from the theoretical and practical perspectives. We first show that the acceptance ratio and step size of this sampler are independent of the overall problem dimension when (i) the target distribution has sparse conditional structure, and (ii) this structure is reflected in the partial updating strategy of MALA-within-Gibbs. If, in addition, the target density is blockwise log-concave, then the sampler's convergence rate is independent of dimension. From a practical perspective, we expect that MALA-within-Gibbs is useful for solving high-dimensional Bayesian inference problems where the posterior exhibits sparse conditional structure at least approximately. In this context, a partitioning of the state that correctly reflects the sparse conditional structure must be found, and we illustrate this process in two numerical examples. We also discuss trade-offs between the block size used for partial updating and computational requirements that may increase with the number of blocks. 2021-10-27T20:23:35Z 2021-10-27T20:23:35Z 2020 2021-05-03T15:07:43Z Article http://purl.org/eprint/type/JournalArticle https://hdl.handle.net/1721.1/135471 en 10.1137/19M1284014 SIAM Journal on Scientific Computing Article is made available in accordance with the publisher's policy and may be subject to US copyright law. Please refer to the publisher's site for terms of use. application/pdf Society for Industrial & Applied Mathematics (SIAM) SIAM |
| spellingShingle | Tong, XT Morzfeld, M Marzouk, YM MALA-within-Gibbs Samplers for High-Dimensional Distributions with Sparse Conditional Structure |
| title | MALA-within-Gibbs Samplers for High-Dimensional Distributions with Sparse Conditional Structure |
| title_full | MALA-within-Gibbs Samplers for High-Dimensional Distributions with Sparse Conditional Structure |
| title_fullStr | MALA-within-Gibbs Samplers for High-Dimensional Distributions with Sparse Conditional Structure |
| title_full_unstemmed | MALA-within-Gibbs Samplers for High-Dimensional Distributions with Sparse Conditional Structure |
| title_short | MALA-within-Gibbs Samplers for High-Dimensional Distributions with Sparse Conditional Structure |
| title_sort | mala within gibbs samplers for high dimensional distributions with sparse conditional structure |
| url | https://hdl.handle.net/1721.1/135471 |
| work_keys_str_mv | AT tongxt malawithingibbssamplersforhighdimensionaldistributionswithsparseconditionalstructure AT morzfeldm malawithingibbssamplersforhighdimensionaldistributionswithsparseconditionalstructure AT marzoukym malawithingibbssamplersforhighdimensionaldistributionswithsparseconditionalstructure |