AutoCyclic: Deep Learning Optimizer for Time Series Data Prediction
Time series prediction poses a formidable challenge, marked by the inherent difficulty in capturing long-term dependencies and adapting to intricate data patterns. Existing methods, spanning statistical models and neural networks, often grapple with issues such as underfitting and overfitting. This...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2024-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10410839/ |
| _version_ | 1797338485177188352 |
|---|---|
| author | Christian Arthur Novanto Yudistira Candra Dewi |
| author_facet | Christian Arthur Novanto Yudistira Candra Dewi |
| author_sort | Christian Arthur |
| collection | DOAJ |
| description | Time series prediction poses a formidable challenge, marked by the inherent difficulty in capturing long-term dependencies and adapting to intricate data patterns. Existing methods, spanning statistical models and neural networks, often grapple with issues such as underfitting and overfitting. This study addresses these challenges by introducing Autocyclic Learning Rate (AutoCyclic), an innovative approach that seamlessly integrates cosine cyclic learning rates with considerations for autocorrelation and variance. AutoCyclic dynamically adjusts learning rates based on the characteristics of time series data, effectively mitigating challenges related to local minima and demonstrating robust adaptability to outliers. In evaluation across diverse datasets, including ETTm2, M4, and WindTurbine, AutoCyclic consistently outperforms traditional optimizers such as Adams Optimizer and Cosine Cyclic Learning Rate. The results underscore AutoCyclic’s superior performance, showcasing its potential as a pivotal tool for enhancing predictive modeling in various time series forecasting scenarios. The groundbreaking nature of AutoCyclic lies in its ability to address the complexities of time series prediction, providing a valuable solution to the limitations faced by existing models. The study serves as a key contribution to the ongoing research in timeseries data prediction, with implications for improving the accuracy and efficiency of predictive models in diverse applications. For those interested in implementing AutoCyclic, the code is available at <uri>https://github.com/wtfish/AutoCyclic</uri>. |
| first_indexed | 2024-03-08T09:31:54Z |
| format | Article |
| id | doaj.art-7246ce2472de420082571fa741168415 |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-03-08T09:31:54Z |
| publishDate | 2024-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-7246ce2472de420082571fa7411684152024-01-31T00:01:28ZengIEEEIEEE Access2169-35362024-01-0112140141402610.1109/ACCESS.2024.335655310410839AutoCyclic: Deep Learning Optimizer for Time Series Data PredictionChristian Arthur0https://orcid.org/0009-0009-8145-1439Novanto Yudistira1https://orcid.org/0000-0001-5330-5930Candra Dewi2Departemen Teknik Informatika, Fakultas Ilmu Komputer, Universitas Brawijaya, Ketawanggede, Lowokwaru, Malang, IndonesiaDepartemen Teknik Informatika, Fakultas Ilmu Komputer, Universitas Brawijaya, Ketawanggede, Lowokwaru, Malang, IndonesiaDepartemen Teknik Informatika, Fakultas Ilmu Komputer, Universitas Brawijaya, Ketawanggede, Lowokwaru, Malang, IndonesiaTime series prediction poses a formidable challenge, marked by the inherent difficulty in capturing long-term dependencies and adapting to intricate data patterns. Existing methods, spanning statistical models and neural networks, often grapple with issues such as underfitting and overfitting. This study addresses these challenges by introducing Autocyclic Learning Rate (AutoCyclic), an innovative approach that seamlessly integrates cosine cyclic learning rates with considerations for autocorrelation and variance. AutoCyclic dynamically adjusts learning rates based on the characteristics of time series data, effectively mitigating challenges related to local minima and demonstrating robust adaptability to outliers. In evaluation across diverse datasets, including ETTm2, M4, and WindTurbine, AutoCyclic consistently outperforms traditional optimizers such as Adams Optimizer and Cosine Cyclic Learning Rate. The results underscore AutoCyclic’s superior performance, showcasing its potential as a pivotal tool for enhancing predictive modeling in various time series forecasting scenarios. The groundbreaking nature of AutoCyclic lies in its ability to address the complexities of time series prediction, providing a valuable solution to the limitations faced by existing models. The study serves as a key contribution to the ongoing research in timeseries data prediction, with implications for improving the accuracy and efficiency of predictive models in diverse applications. For those interested in implementing AutoCyclic, the code is available at <uri>https://github.com/wtfish/AutoCyclic</uri>.https://ieeexplore.ieee.org/document/10410839/Autocorrelationcosine cyclic learning ratedeep transformeroptimizertime seriesvariance |
| spellingShingle | Christian Arthur Novanto Yudistira Candra Dewi AutoCyclic: Deep Learning Optimizer for Time Series Data Prediction IEEE Access Autocorrelation cosine cyclic learning rate deep transformer optimizer time series variance |
| title | AutoCyclic: Deep Learning Optimizer for Time Series Data Prediction |
| title_full | AutoCyclic: Deep Learning Optimizer for Time Series Data Prediction |
| title_fullStr | AutoCyclic: Deep Learning Optimizer for Time Series Data Prediction |
| title_full_unstemmed | AutoCyclic: Deep Learning Optimizer for Time Series Data Prediction |
| title_short | AutoCyclic: Deep Learning Optimizer for Time Series Data Prediction |
| title_sort | autocyclic deep learning optimizer for time series data prediction |
| topic | Autocorrelation cosine cyclic learning rate deep transformer optimizer time series variance |
| url | https://ieeexplore.ieee.org/document/10410839/ |
| work_keys_str_mv | AT christianarthur autocyclicdeeplearningoptimizerfortimeseriesdataprediction AT novantoyudistira autocyclicdeeplearningoptimizerfortimeseriesdataprediction AT candradewi autocyclicdeeplearningoptimizerfortimeseriesdataprediction |