Enhancing Performance of a Deep Neural Network: A Comparative Analysis of Optimization Algorithms
Adopting the most suitable optimization algorithm (optimizer) for a Neural Network Model is among the most important ventures in Deep Learning and all classes of Neural Networks. It’s a case of trial and error experimentation. In this paper, we will experiment with seven of the most popular optimiza...
| Main Author: | |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Ediciones Universidad de Salamanca
2020-06-01
|
| Series: | Advances in Distributed Computing and Artificial Intelligence Journal |
| Subjects: | |
| Online Access: | https://revistas.usal.es/index.php/2255-2863/article/view/23727 |
| _version_ | 1818663462215090176 |
|---|---|
| author | Noor Fatima |
| author_facet | Noor Fatima |
| author_sort | Noor Fatima |
| collection | DOAJ |
| description | Adopting the most suitable optimization algorithm (optimizer) for a Neural Network Model is among the most important ventures in Deep Learning and all classes of Neural Networks. It’s a case of trial and error experimentation. In this paper, we will experiment with seven of the most popular optimization algorithms namely: sgd, rmsprop, adagrad, adadelta, adam, adamax and nadam on four unrelated datasets discretely, to conclude which one dispenses the best accuracy, efficiency and performance to our deep neural network. This work will provide insightful analysis to a data scientist in choosing the best optimizer while modelling their deep neural network. |
| first_indexed | 2024-12-17T05:17:14Z |
| format | Article |
| id | doaj.art-7bef20e8ddad46ce96219c15256ce242 |
| institution | Directory Open Access Journal |
| issn | 2255-2863 |
| language | English |
| last_indexed | 2024-12-17T05:17:14Z |
| publishDate | 2020-06-01 |
| publisher | Ediciones Universidad de Salamanca |
| record_format | Article |
| series | Advances in Distributed Computing and Artificial Intelligence Journal |
| spelling | doaj.art-7bef20e8ddad46ce96219c15256ce2422022-12-21T22:02:04ZengEdiciones Universidad de SalamancaAdvances in Distributed Computing and Artificial Intelligence Journal2255-28632020-06-0192799010.14201/ADCAIJ202092799023727Enhancing Performance of a Deep Neural Network: A Comparative Analysis of Optimization AlgorithmsNoor Fatima0Aligarh Muslim UniversityAdopting the most suitable optimization algorithm (optimizer) for a Neural Network Model is among the most important ventures in Deep Learning and all classes of Neural Networks. It’s a case of trial and error experimentation. In this paper, we will experiment with seven of the most popular optimization algorithms namely: sgd, rmsprop, adagrad, adadelta, adam, adamax and nadam on four unrelated datasets discretely, to conclude which one dispenses the best accuracy, efficiency and performance to our deep neural network. This work will provide insightful analysis to a data scientist in choosing the best optimizer while modelling their deep neural network.https://revistas.usal.es/index.php/2255-2863/article/view/23727adadeltaadagradadamadamaxdeep learningneural networksnadamoptimization algorithmsrmspropsgd |
| spellingShingle | Noor Fatima Enhancing Performance of a Deep Neural Network: A Comparative Analysis of Optimization Algorithms Advances in Distributed Computing and Artificial Intelligence Journal adadelta adagrad adam adamax deep learning neural networks nadam optimization algorithms rmsprop sgd |
| title | Enhancing Performance of a Deep Neural Network: A Comparative Analysis of Optimization Algorithms |
| title_full | Enhancing Performance of a Deep Neural Network: A Comparative Analysis of Optimization Algorithms |
| title_fullStr | Enhancing Performance of a Deep Neural Network: A Comparative Analysis of Optimization Algorithms |
| title_full_unstemmed | Enhancing Performance of a Deep Neural Network: A Comparative Analysis of Optimization Algorithms |
| title_short | Enhancing Performance of a Deep Neural Network: A Comparative Analysis of Optimization Algorithms |
| title_sort | enhancing performance of a deep neural network a comparative analysis of optimization algorithms |
| topic | adadelta adagrad adam adamax deep learning neural networks nadam optimization algorithms rmsprop sgd |
| url | https://revistas.usal.es/index.php/2255-2863/article/view/23727 |
| work_keys_str_mv | AT noorfatima enhancingperformanceofadeepneuralnetworkacomparativeanalysisofoptimizationalgorithms |