Adaptive Magnetic Hamiltonian Monte Carlo
Magnetic Hamiltonian Monte Carlo (MHMC) is a Markov Chain Monte Carlo method that expands on Hamiltonian Monte Carlo (HMC) by adding a magnetic field to Hamiltonian dynamics. This magnetic field offers a great deal of flexibility over HMC and encourages more efficient exploration of the target poste...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2021-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/9614186/ |
| _version_ | 1818345189461196800 |
|---|---|
| author | Wilson Tsakane Mongwe Rendani Mbuvha Tshilidzi Marwala |
| author_facet | Wilson Tsakane Mongwe Rendani Mbuvha Tshilidzi Marwala |
| author_sort | Wilson Tsakane Mongwe |
| collection | DOAJ |
| description | Magnetic Hamiltonian Monte Carlo (MHMC) is a Markov Chain Monte Carlo method that expands on Hamiltonian Monte Carlo (HMC) by adding a magnetic field to Hamiltonian dynamics. This magnetic field offers a great deal of flexibility over HMC and encourages more efficient exploration of the target posterior. This results in faster convergence and lower autocorrelations in the generated samples compared to HMC. However, as with HMC, MHMC is sensitive to the user specified trajectory length and step size. Automatically setting the parameters of MHMC is yet to be considered in the literature. In this work, we present the Adaptive MHMC (A-MHMC) algorithm which extends MHMC in that it automatically sets the parameters of MHMC and thus eliminates the need for the user to manually set a trajectory length and step size. The trajectory length adaptation is based on an extension of the No-U-Turn Sampler (NUTS) methodology to incorporate the magnetic field present in MHMC, while the step size is set via dual averaging during the burn-in period. Empirical results based on experiments performed on jump diffusion processes calibrated to real world financial market data, a simulation study using multivariate Gaussian distributions and real world benchmark datasets modelled using Bayesian Logistic Regression show that A-MHMC outperforms MHMC and NUTS on an effective sample size basis. In addition, A-MHMC provides significant relative speed up (up to 40 times) over MHMC and produces similar time normalised effective samples sizes relative to NUTS. |
| first_indexed | 2024-12-13T16:58:25Z |
| format | Article |
| id | doaj.art-11741b05895b4cef99bef5d3b1f5ec8a |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-12-13T16:58:25Z |
| publishDate | 2021-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-11741b05895b4cef99bef5d3b1f5ec8a2022-12-21T23:37:52ZengIEEEIEEE Access2169-35362021-01-01915299315300310.1109/ACCESS.2021.31279319614186Adaptive Magnetic Hamiltonian Monte CarloWilson Tsakane Mongwe0https://orcid.org/0000-0003-2832-3584Rendani Mbuvha1https://orcid.org/0000-0002-7337-9176Tshilidzi Marwala2School of Electrical Engineering, University of Johannesburg, Auckland Park, South AfricaSchool of Statistics and Actuarial Science, University of Witwatersrand, Johannesburg, South AfricaSchool of Electrical Engineering, University of Johannesburg, Auckland Park, South AfricaMagnetic Hamiltonian Monte Carlo (MHMC) is a Markov Chain Monte Carlo method that expands on Hamiltonian Monte Carlo (HMC) by adding a magnetic field to Hamiltonian dynamics. This magnetic field offers a great deal of flexibility over HMC and encourages more efficient exploration of the target posterior. This results in faster convergence and lower autocorrelations in the generated samples compared to HMC. However, as with HMC, MHMC is sensitive to the user specified trajectory length and step size. Automatically setting the parameters of MHMC is yet to be considered in the literature. In this work, we present the Adaptive MHMC (A-MHMC) algorithm which extends MHMC in that it automatically sets the parameters of MHMC and thus eliminates the need for the user to manually set a trajectory length and step size. The trajectory length adaptation is based on an extension of the No-U-Turn Sampler (NUTS) methodology to incorporate the magnetic field present in MHMC, while the step size is set via dual averaging during the burn-in period. Empirical results based on experiments performed on jump diffusion processes calibrated to real world financial market data, a simulation study using multivariate Gaussian distributions and real world benchmark datasets modelled using Bayesian Logistic Regression show that A-MHMC outperforms MHMC and NUTS on an effective sample size basis. In addition, A-MHMC provides significant relative speed up (up to 40 times) over MHMC and produces similar time normalised effective samples sizes relative to NUTS.https://ieeexplore.ieee.org/document/9614186/Bayesian logistic regressionjump diffusion processesadaptiveMagnetic Hamiltonian Monte CarloMarkov Chain Monte Carlo |
| spellingShingle | Wilson Tsakane Mongwe Rendani Mbuvha Tshilidzi Marwala Adaptive Magnetic Hamiltonian Monte Carlo IEEE Access Bayesian logistic regression jump diffusion processes adaptive Magnetic Hamiltonian Monte Carlo Markov Chain Monte Carlo |
| title | Adaptive Magnetic Hamiltonian Monte Carlo |
| title_full | Adaptive Magnetic Hamiltonian Monte Carlo |
| title_fullStr | Adaptive Magnetic Hamiltonian Monte Carlo |
| title_full_unstemmed | Adaptive Magnetic Hamiltonian Monte Carlo |
| title_short | Adaptive Magnetic Hamiltonian Monte Carlo |
| title_sort | adaptive magnetic hamiltonian monte carlo |
| topic | Bayesian logistic regression jump diffusion processes adaptive Magnetic Hamiltonian Monte Carlo Markov Chain Monte Carlo |
| url | https://ieeexplore.ieee.org/document/9614186/ |
| work_keys_str_mv | AT wilsontsakanemongwe adaptivemagnetichamiltonianmontecarlo AT rendanimbuvha adaptivemagnetichamiltonianmontecarlo AT tshilidzimarwala adaptivemagnetichamiltonianmontecarlo |