Temperature based restricted boltzmann machines
Restricted Boltzmann machines (RBMs), which apply graphical models to learning probability distribution over a set of inputs, have attracted much attention recently since being proposed as building blocks of multi-layer learning systems called deep belief networks (DBNs). Note that temperature is a...
| Main Authors: | , , , , , , |
|---|---|
| Other Authors: | |
| Format: | Journal Article |
| Language: | English |
| Published: |
2018
|
| Subjects: | |
| Online Access: | https://hdl.handle.net/10356/87524 http://hdl.handle.net/10220/46750 |
| _version_ | 1826120312210587648 |
|---|---|
| author | Li, Guoqi Deng, Lei Xu, Yi Wen, Changyun Wang, Wei Pei, Jing Shi, Luping |
| author2 | School of Computer Science and Engineering |
| author_facet | School of Computer Science and Engineering Li, Guoqi Deng, Lei Xu, Yi Wen, Changyun Wang, Wei Pei, Jing Shi, Luping |
| author_sort | Li, Guoqi |
| collection | NTU |
| description | Restricted Boltzmann machines (RBMs), which apply graphical models to learning probability distribution over a set of inputs, have attracted much attention recently since being proposed as building blocks of multi-layer learning systems called deep belief networks (DBNs). Note that temperature is a key factor of the Boltzmann distribution that RBMs originate from. However, none of existing schemes have considered the impact of temperature in the graphical model of DBNs. In this work, we propose temperature based restricted Boltzmann machines (TRBMs) which reveals that temperature is an essential parameter controlling the selectivity of the firing neurons in the hidden layers. We theoretically prove that the effect of temperature can be adjusted by setting the parameter of the sharpness of the logistic function in the proposed TRBMs. The performance of RBMs can be improved by adjusting the temperature parameter of TRBMs. This work provides a comprehensive insights into the deep belief networks and deep learning architectures from a physical point of view. |
| first_indexed | 2024-10-01T05:14:25Z |
| format | Journal Article |
| id | ntu-10356/87524 |
| institution | Nanyang Technological University |
| language | English |
| last_indexed | 2024-10-01T05:14:25Z |
| publishDate | 2018 |
| record_format | dspace |
| spelling | ntu-10356/875242022-02-16T16:30:46Z Temperature based restricted boltzmann machines Li, Guoqi Deng, Lei Xu, Yi Wen, Changyun Wang, Wei Pei, Jing Shi, Luping School of Computer Science and Engineering School of Electrical and Electronic Engineering Restricted Boltzmann Machines DRNTU::Engineering::Electrical and electronic engineering Temperature Restricted Boltzmann machines (RBMs), which apply graphical models to learning probability distribution over a set of inputs, have attracted much attention recently since being proposed as building blocks of multi-layer learning systems called deep belief networks (DBNs). Note that temperature is a key factor of the Boltzmann distribution that RBMs originate from. However, none of existing schemes have considered the impact of temperature in the graphical model of DBNs. In this work, we propose temperature based restricted Boltzmann machines (TRBMs) which reveals that temperature is an essential parameter controlling the selectivity of the firing neurons in the hidden layers. We theoretically prove that the effect of temperature can be adjusted by setting the parameter of the sharpness of the logistic function in the proposed TRBMs. The performance of RBMs can be improved by adjusting the temperature parameter of TRBMs. This work provides a comprehensive insights into the deep belief networks and deep learning architectures from a physical point of view. Published version 2018-11-30T05:28:15Z 2019-12-06T16:43:43Z 2018-11-30T05:28:15Z 2019-12-06T16:43:43Z 2016 Journal Article Li, G., Deng, L., Xu, Y., Wen, C., Wang, W., Pei, J., & Shi, L. (2016). Temperature based restricted boltzmann machines. Scientific Reports, 6, 19133-. doi:10.1038/srep19133 https://hdl.handle.net/10356/87524 http://hdl.handle.net/10220/46750 10.1038/srep19133 26758235 en Scientific Reports © 2016 The Authors (Nature Publishing Group). This work is licensed under a Creative Commons Attribution 4.0 International License. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in the credit line; if the material is not included under the Creative Commons license, users will need to obtain permission from the license holder to reproduce the material. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/ 12 p. application/pdf |
| spellingShingle | Restricted Boltzmann Machines DRNTU::Engineering::Electrical and electronic engineering Temperature Li, Guoqi Deng, Lei Xu, Yi Wen, Changyun Wang, Wei Pei, Jing Shi, Luping Temperature based restricted boltzmann machines |
| title | Temperature based restricted boltzmann machines |
| title_full | Temperature based restricted boltzmann machines |
| title_fullStr | Temperature based restricted boltzmann machines |
| title_full_unstemmed | Temperature based restricted boltzmann machines |
| title_short | Temperature based restricted boltzmann machines |
| title_sort | temperature based restricted boltzmann machines |
| topic | Restricted Boltzmann Machines DRNTU::Engineering::Electrical and electronic engineering Temperature |
| url | https://hdl.handle.net/10356/87524 http://hdl.handle.net/10220/46750 |
| work_keys_str_mv | AT liguoqi temperaturebasedrestrictedboltzmannmachines AT denglei temperaturebasedrestrictedboltzmannmachines AT xuyi temperaturebasedrestrictedboltzmannmachines AT wenchangyun temperaturebasedrestrictedboltzmannmachines AT wangwei temperaturebasedrestrictedboltzmannmachines AT peijing temperaturebasedrestrictedboltzmannmachines AT shiluping temperaturebasedrestrictedboltzmannmachines |