Fast Sampling of Score-Based Models With Cyclical Diffusion Sampling
Diffusion models have recently exhibited significant potential in generative modeling, surpassing generative adversarial networks concerning perceptual quality and autoregressive models in density estimation. However, a notable drawback of these models is their slow sampling time, requiring numerous...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2024-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10433190/ |
| _version_ | 1797213599994740736 |
|---|---|
| author | Karimul Makhtidi Alhadi Bustamam Risman Adnan Hanif Amal Robbani Wibowo Mangunwardoyo Mohammad Asif Khan |
| author_facet | Karimul Makhtidi Alhadi Bustamam Risman Adnan Hanif Amal Robbani Wibowo Mangunwardoyo Mohammad Asif Khan |
| author_sort | Karimul Makhtidi |
| collection | DOAJ |
| description | Diffusion models have recently exhibited significant potential in generative modeling, surpassing generative adversarial networks concerning perceptual quality and autoregressive models in density estimation. However, a notable drawback of these models is their slow sampling time, requiring numerous model evaluations to generate high-quality samples. This research proposes a technique termed cyclical diffusion sampling that incorporates cyclical stochastic gradient Langevin dynamics (SGLD) with elucidated diffusion models (EDM) sampler to enhance stability when utilizing a limited number of sampling steps. Cyclical step-size scheduling has been demonstrated to enhance the effectiveness of SGLD in learning complex multimodal distributions. We have shown, via empirical assessments, that cyclical diffusion sampling significantly enhances image quality and markedly decreases inference time. Notably, the method preserves simplicity and requires no alterations to the network architecture, which promotes straightforward reproducibility and seamless integration with current methodologies. |
| first_indexed | 2024-04-24T11:00:51Z |
| format | Article |
| id | doaj.art-7b5912a4b53d4ea9855df71612a5ac80 |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-04-24T11:00:51Z |
| publishDate | 2024-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-7b5912a4b53d4ea9855df71612a5ac802024-04-11T23:00:44ZengIEEEIEEE Access2169-35362024-01-0112495784958910.1109/ACCESS.2024.336514610433190Fast Sampling of Score-Based Models With Cyclical Diffusion SamplingKarimul Makhtidi0Alhadi Bustamam1https://orcid.org/0000-0002-7408-074XRisman Adnan2Hanif Amal Robbani3https://orcid.org/0000-0002-1510-6559Wibowo Mangunwardoyo4Mohammad Asif Khan5Department of Mathematics, Universitas Indonesia, Depok City, IndonesiaDepartment of Mathematics, Universitas Indonesia, Depok City, IndonesiaKalbe Digital Laboratory, Jakarta, IndonesiaDepartment of Mathematics, Universitas Indonesia, Depok City, IndonesiaDepartment of Biology, Universitas Indonesia, Depok City, IndonesiaCollege of Computing and Information Technology, University of Doha for Science and Technology, Doha, QatarDiffusion models have recently exhibited significant potential in generative modeling, surpassing generative adversarial networks concerning perceptual quality and autoregressive models in density estimation. However, a notable drawback of these models is their slow sampling time, requiring numerous model evaluations to generate high-quality samples. This research proposes a technique termed cyclical diffusion sampling that incorporates cyclical stochastic gradient Langevin dynamics (SGLD) with elucidated diffusion models (EDM) sampler to enhance stability when utilizing a limited number of sampling steps. Cyclical step-size scheduling has been demonstrated to enhance the effectiveness of SGLD in learning complex multimodal distributions. We have shown, via empirical assessments, that cyclical diffusion sampling significantly enhances image quality and markedly decreases inference time. Notably, the method preserves simplicity and requires no alterations to the network architecture, which promotes straightforward reproducibility and seamless integration with current methodologies.https://ieeexplore.ieee.org/document/10433190/Score-based generative modelsdenoising diffusion probabilistic modelsLangevin dynamicsstochastic differential equation |
| spellingShingle | Karimul Makhtidi Alhadi Bustamam Risman Adnan Hanif Amal Robbani Wibowo Mangunwardoyo Mohammad Asif Khan Fast Sampling of Score-Based Models With Cyclical Diffusion Sampling IEEE Access Score-based generative models denoising diffusion probabilistic models Langevin dynamics stochastic differential equation |
| title | Fast Sampling of Score-Based Models With Cyclical Diffusion Sampling |
| title_full | Fast Sampling of Score-Based Models With Cyclical Diffusion Sampling |
| title_fullStr | Fast Sampling of Score-Based Models With Cyclical Diffusion Sampling |
| title_full_unstemmed | Fast Sampling of Score-Based Models With Cyclical Diffusion Sampling |
| title_short | Fast Sampling of Score-Based Models With Cyclical Diffusion Sampling |
| title_sort | fast sampling of score based models with cyclical diffusion sampling |
| topic | Score-based generative models denoising diffusion probabilistic models Langevin dynamics stochastic differential equation |
| url | https://ieeexplore.ieee.org/document/10433190/ |
| work_keys_str_mv | AT karimulmakhtidi fastsamplingofscorebasedmodelswithcyclicaldiffusionsampling AT alhadibustamam fastsamplingofscorebasedmodelswithcyclicaldiffusionsampling AT rismanadnan fastsamplingofscorebasedmodelswithcyclicaldiffusionsampling AT hanifamalrobbani fastsamplingofscorebasedmodelswithcyclicaldiffusionsampling AT wibowomangunwardoyo fastsamplingofscorebasedmodelswithcyclicaldiffusionsampling AT mohammadasifkhan fastsamplingofscorebasedmodelswithcyclicaldiffusionsampling |