Accelerated Diffusion-Based Sampling by the Non-Reversible Dynamics with Skew-Symmetric Matrices
Langevin dynamics (LD) has been extensively studied theoretically and practically as a basic sampling technique. Recently, the incorporation of non-reversible dynamics into LD is attracting attention because it accelerates the mixing speed of LD. Popular choices for non-reversible dynamics include u...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2021-07-01
|
| Series: | Entropy |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1099-4300/23/8/993 |
| _version_ | 1827685268032847872 |
|---|---|
| author | Futoshi Futami Tomoharu Iwata Naonori Ueda Issei Sato |
| author_facet | Futoshi Futami Tomoharu Iwata Naonori Ueda Issei Sato |
| author_sort | Futoshi Futami |
| collection | DOAJ |
| description | Langevin dynamics (LD) has been extensively studied theoretically and practically as a basic sampling technique. Recently, the incorporation of non-reversible dynamics into LD is attracting attention because it accelerates the mixing speed of LD. Popular choices for non-reversible dynamics include underdamped Langevin dynamics (ULD), which uses second-order dynamics and perturbations with skew-symmetric matrices. Although ULD has been widely used in practice, the application of skew acceleration is limited although it is expected to show superior performance theoretically. Current work lacks a theoretical understanding of issues that are important to practitioners, including the selection criteria for skew-symmetric matrices, quantitative evaluations of acceleration, and the large memory cost of storing skew matrices. In this study, we theoretically and numerically clarify these problems by analyzing acceleration focusing on how the skew-symmetric matrix perturbs the Hessian matrix of potential functions. We also present a practical algorithm that accelerates the standard LD and ULD, which uses novel memory-efficient skew-symmetric matrices under parallel-chain Monte Carlo settings. |
| first_indexed | 2024-03-10T08:49:27Z |
| format | Article |
| id | doaj.art-22039549696b432f900f76091b684484 |
| institution | Directory Open Access Journal |
| issn | 1099-4300 |
| language | English |
| last_indexed | 2024-03-10T08:49:27Z |
| publishDate | 2021-07-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Entropy |
| spelling | doaj.art-22039549696b432f900f76091b6844842023-11-22T07:34:44ZengMDPI AGEntropy1099-43002021-07-0123899310.3390/e23080993Accelerated Diffusion-Based Sampling by the Non-Reversible Dynamics with Skew-Symmetric MatricesFutoshi Futami0Tomoharu Iwata1Naonori Ueda2Issei Sato3Communication Science Laboratories, NTT, Hikaridai, Seika-cho, “Keihanna Science City”, Kyoto 619-0237, JapanCommunication Science Laboratories, NTT, Hikaridai, Seika-cho, “Keihanna Science City”, Kyoto 619-0237, JapanCommunication Science Laboratories, NTT, Hikaridai, Seika-cho, “Keihanna Science City”, Kyoto 619-0237, JapanDepartment of Computer Science, Graduate School of Information Science and Technology, The University of Tokyo, 7-3-1 Hongo, Bunkyo-ku, Tokyo 113-0033, JapanLangevin dynamics (LD) has been extensively studied theoretically and practically as a basic sampling technique. Recently, the incorporation of non-reversible dynamics into LD is attracting attention because it accelerates the mixing speed of LD. Popular choices for non-reversible dynamics include underdamped Langevin dynamics (ULD), which uses second-order dynamics and perturbations with skew-symmetric matrices. Although ULD has been widely used in practice, the application of skew acceleration is limited although it is expected to show superior performance theoretically. Current work lacks a theoretical understanding of issues that are important to practitioners, including the selection criteria for skew-symmetric matrices, quantitative evaluations of acceleration, and the large memory cost of storing skew matrices. In this study, we theoretically and numerically clarify these problems by analyzing acceleration focusing on how the skew-symmetric matrix perturbs the Hessian matrix of potential functions. We also present a practical algorithm that accelerates the standard LD and ULD, which uses novel memory-efficient skew-symmetric matrices under parallel-chain Monte Carlo settings.https://www.mdpi.com/1099-4300/23/8/993Markov Chain Monte CarloLangevin dynamicsHamilton Monte Carlonon-reversible dynamics |
| spellingShingle | Futoshi Futami Tomoharu Iwata Naonori Ueda Issei Sato Accelerated Diffusion-Based Sampling by the Non-Reversible Dynamics with Skew-Symmetric Matrices Entropy Markov Chain Monte Carlo Langevin dynamics Hamilton Monte Carlo non-reversible dynamics |
| title | Accelerated Diffusion-Based Sampling by the Non-Reversible Dynamics with Skew-Symmetric Matrices |
| title_full | Accelerated Diffusion-Based Sampling by the Non-Reversible Dynamics with Skew-Symmetric Matrices |
| title_fullStr | Accelerated Diffusion-Based Sampling by the Non-Reversible Dynamics with Skew-Symmetric Matrices |
| title_full_unstemmed | Accelerated Diffusion-Based Sampling by the Non-Reversible Dynamics with Skew-Symmetric Matrices |
| title_short | Accelerated Diffusion-Based Sampling by the Non-Reversible Dynamics with Skew-Symmetric Matrices |
| title_sort | accelerated diffusion based sampling by the non reversible dynamics with skew symmetric matrices |
| topic | Markov Chain Monte Carlo Langevin dynamics Hamilton Monte Carlo non-reversible dynamics |
| url | https://www.mdpi.com/1099-4300/23/8/993 |
| work_keys_str_mv | AT futoshifutami accelerateddiffusionbasedsamplingbythenonreversibledynamicswithskewsymmetricmatrices AT tomoharuiwata accelerateddiffusionbasedsamplingbythenonreversibledynamicswithskewsymmetricmatrices AT naonoriueda accelerateddiffusionbasedsamplingbythenonreversibledynamicswithskewsymmetricmatrices AT isseisato accelerateddiffusionbasedsamplingbythenonreversibledynamicswithskewsymmetricmatrices |