A study of acceleration techniques in training neural networks - local adaptive techniques / Norpah Mahat
Till today, it has been a great challenge in optimizing the training time in neural networks. This paper presents Local Adaptive Techniques and Dynamic Adaptation Methods as acceleration techniques for neural networks. The first technique is based on weight-specific information such as the temporal...
| Main Author: | |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Universiti Teknologi MARA, Pulau Pinang & Pusat Penerbitan Universiti (UPENA)
2011
|
| Subjects: | |
| Online Access: | https://ir.uitm.edu.my/id/eprint/8852/1/AJ_NORPAH%20MAHAT%20ESTEEM%2011.pdf |
| _version_ | 1825733854655152128 |
|---|---|
| author | Mahat, Norpah |
| author_facet | Mahat, Norpah |
| author_sort | Mahat, Norpah |
| collection | UITM |
| description | Till today, it has been a great challenge in optimizing the training time in neural networks. This paper presents Local Adaptive Techniques and Dynamic Adaptation Methods as acceleration techniques for neural networks. The first technique is based on weight-specific information such as the temporal behavior of the partial derivative of the current weight. The second technique is dynamically adapts the momentum factor, a, and learning rate, ᶯ with respect to the iteration number or gradient. Some of the most popular learning algorithms are described and discussed. Simulations on a real world application problem are conducted to evaluate and compare the performance of a local adaptive strategies with various popular training algorithms include global adaptive strategies. These techniques have been compared and measured in terms of
gradient and error function evaluations, and percentage of success. |
| first_indexed | 2024-03-06T01:23:14Z |
| format | Article |
| id | oai:ir.uitm.edu.my:8852 |
| institution | Universiti Teknologi MARA |
| language | English |
| last_indexed | 2024-03-06T01:23:14Z |
| publishDate | 2011 |
| publisher | Universiti Teknologi MARA, Pulau Pinang & Pusat Penerbitan Universiti (UPENA) |
| record_format | dspace |
| spelling | oai:ir.uitm.edu.my:88522017-03-14T04:42:42Z https://ir.uitm.edu.my/id/eprint/8852/ A study of acceleration techniques in training neural networks - local adaptive techniques / Norpah Mahat esteem Mahat, Norpah Electronic Computers. Computer Science Till today, it has been a great challenge in optimizing the training time in neural networks. This paper presents Local Adaptive Techniques and Dynamic Adaptation Methods as acceleration techniques for neural networks. The first technique is based on weight-specific information such as the temporal behavior of the partial derivative of the current weight. The second technique is dynamically adapts the momentum factor, a, and learning rate, ᶯ with respect to the iteration number or gradient. Some of the most popular learning algorithms are described and discussed. Simulations on a real world application problem are conducted to evaluate and compare the performance of a local adaptive strategies with various popular training algorithms include global adaptive strategies. These techniques have been compared and measured in terms of gradient and error function evaluations, and percentage of success. Universiti Teknologi MARA, Pulau Pinang & Pusat Penerbitan Universiti (UPENA) 2011 Article PeerReviewed text en https://ir.uitm.edu.my/id/eprint/8852/1/AJ_NORPAH%20MAHAT%20ESTEEM%2011.pdf A study of acceleration techniques in training neural networks - local adaptive techniques / Norpah Mahat. (2011) Esteem Academic Journal <https://ir.uitm.edu.my/view/publication/Esteem_Academic_Journal/>, 7 (2). pp. 35-51. ISSN 1675-7939 |
| spellingShingle | Electronic Computers. Computer Science Mahat, Norpah A study of acceleration techniques in training neural networks - local adaptive techniques / Norpah Mahat |
| title | A study of acceleration techniques in training neural networks - local adaptive techniques / Norpah Mahat |
| title_full | A study of acceleration techniques in training neural networks - local adaptive techniques / Norpah Mahat |
| title_fullStr | A study of acceleration techniques in training neural networks - local adaptive techniques / Norpah Mahat |
| title_full_unstemmed | A study of acceleration techniques in training neural networks - local adaptive techniques / Norpah Mahat |
| title_short | A study of acceleration techniques in training neural networks - local adaptive techniques / Norpah Mahat |
| title_sort | study of acceleration techniques in training neural networks local adaptive techniques norpah mahat |
| topic | Electronic Computers. Computer Science |
| url | https://ir.uitm.edu.my/id/eprint/8852/1/AJ_NORPAH%20MAHAT%20ESTEEM%2011.pdf |
| work_keys_str_mv | AT mahatnorpah astudyofaccelerationtechniquesintrainingneuralnetworkslocaladaptivetechniquesnorpahmahat AT mahatnorpah studyofaccelerationtechniquesintrainingneuralnetworkslocaladaptivetechniquesnorpahmahat |