Combining Optimization Methods Using an Adaptive Meta Optimizer
Optimization methods are of great importance for the efficient training of neural networks. There are many articles in the literature that propose particular variants of existing optimizers. In our article, we propose the use of the combination of two very different optimizers that, when used simult...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2021-06-01
|
| Series: | Algorithms |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1999-4893/14/6/186 |
| _version_ | 1797529536600997888 |
|---|---|
| author | Nicola Landro Ignazio Gallo Riccardo La Grassa |
| author_facet | Nicola Landro Ignazio Gallo Riccardo La Grassa |
| author_sort | Nicola Landro |
| collection | DOAJ |
| description | Optimization methods are of great importance for the efficient training of neural networks. There are many articles in the literature that propose particular variants of existing optimizers. In our article, we propose the use of the combination of two very different optimizers that, when used simultaneously, can exceed the performance of the single optimizers in very different problems. We propose a new optimizer called ATMO (AdapTive Meta Optimizers), which integrates two different optimizers simultaneously weighing the contributions of both. Rather than trying to improve each single one, we leverage both at the same time, as a meta-optimizer, by taking the best of both. We have conducted several experiments on the classification of images and text documents, using various types of deep neural models, and we have demonstrated through experiments that the proposed ATMO produces better performance than the single optimizers. |
| first_indexed | 2024-03-10T10:15:14Z |
| format | Article |
| id | doaj.art-0e910ced08fe4377a40540bd1a116aca |
| institution | Directory Open Access Journal |
| issn | 1999-4893 |
| language | English |
| last_indexed | 2024-03-10T10:15:14Z |
| publishDate | 2021-06-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Algorithms |
| spelling | doaj.art-0e910ced08fe4377a40540bd1a116aca2023-11-22T00:49:57ZengMDPI AGAlgorithms1999-48932021-06-0114618610.3390/a14060186Combining Optimization Methods Using an Adaptive Meta OptimizerNicola Landro0Ignazio Gallo1Riccardo La Grassa2Department of Theoretical and Applied Sciences, University of Insubria, 21100 Varese, ItalyDepartment of Theoretical and Applied Sciences, University of Insubria, 21100 Varese, ItalyDepartment of Theoretical and Applied Sciences, University of Insubria, 21100 Varese, ItalyOptimization methods are of great importance for the efficient training of neural networks. There are many articles in the literature that propose particular variants of existing optimizers. In our article, we propose the use of the combination of two very different optimizers that, when used simultaneously, can exceed the performance of the single optimizers in very different problems. We propose a new optimizer called ATMO (AdapTive Meta Optimizers), which integrates two different optimizers simultaneously weighing the contributions of both. Rather than trying to improve each single one, we leverage both at the same time, as a meta-optimizer, by taking the best of both. We have conducted several experiments on the classification of images and text documents, using various types of deep neural models, and we have demonstrated through experiments that the proposed ATMO produces better performance than the single optimizers.https://www.mdpi.com/1999-4893/14/6/186deep learningoptimization algorithmoptimizerstext classificationimage classification |
| spellingShingle | Nicola Landro Ignazio Gallo Riccardo La Grassa Combining Optimization Methods Using an Adaptive Meta Optimizer Algorithms deep learning optimization algorithm optimizers text classification image classification |
| title | Combining Optimization Methods Using an Adaptive Meta Optimizer |
| title_full | Combining Optimization Methods Using an Adaptive Meta Optimizer |
| title_fullStr | Combining Optimization Methods Using an Adaptive Meta Optimizer |
| title_full_unstemmed | Combining Optimization Methods Using an Adaptive Meta Optimizer |
| title_short | Combining Optimization Methods Using an Adaptive Meta Optimizer |
| title_sort | combining optimization methods using an adaptive meta optimizer |
| topic | deep learning optimization algorithm optimizers text classification image classification |
| url | https://www.mdpi.com/1999-4893/14/6/186 |
| work_keys_str_mv | AT nicolalandro combiningoptimizationmethodsusinganadaptivemetaoptimizer AT ignaziogallo combiningoptimizationmethodsusinganadaptivemetaoptimizer AT riccardolagrassa combiningoptimizationmethodsusinganadaptivemetaoptimizer |