A Convolutional Transformer Model for Multivariate Time Series Prediction
This paper presents a multivariate time series prediction framework based on a transformer model consisting of convolutional neural networks (CNNs). The proposed model has a structure that extracts temporal features of input data through CNN and interprets correlations between variables through an a...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2022-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/9874747/ |
| _version_ | 1811229958627917824 |
|---|---|
| author | Dong-Keon Kim Kwangsu Kim |
| author_facet | Dong-Keon Kim Kwangsu Kim |
| author_sort | Dong-Keon Kim |
| collection | DOAJ |
| description | This paper presents a multivariate time series prediction framework based on a transformer model consisting of convolutional neural networks (CNNs). The proposed model has a structure that extracts temporal features of input data through CNN and interprets correlations between variables through an attention mechanism. This framework solves the problem of the inability to simultaneously analyze the temporal features of the input data and the correlation between variables, which is a limitation of the forecasting models presented in existing studies. We designed a forecasting experiment using several time series datasets with various data characteristics to precisely evaluate the proposed model. In addition, comparative experiments were performed between the proposed model and several predictive models proposed in recent studies. Furthermore, we conducted ablation studies on the extent to which the proposed CNN structure in the prediction model affects the forecasting results by substituting a specific layer of the model. The results of the experiments showed that the proposed predictive model exhibited good performance in predicting time series data with a clear cycle and high correlation between variables, and improved the accuracy by approximately 3% to 5% compared with that of previous studies’ time series prediction models. |
| first_indexed | 2024-04-12T10:22:04Z |
| format | Article |
| id | doaj.art-e9b5dcbc3a6343dbad04cafc79035422 |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-04-12T10:22:04Z |
| publishDate | 2022-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-e9b5dcbc3a6343dbad04cafc790354222022-12-22T03:37:04ZengIEEEIEEE Access2169-35362022-01-011010131910132910.1109/ACCESS.2022.32034169874747A Convolutional Transformer Model for Multivariate Time Series PredictionDong-Keon Kim0Kwangsu Kim1Department of Software, Sungkyunkwan University, Suwon, South KoreaDepartment of Software, Sungkyunkwan University, Suwon, South KoreaThis paper presents a multivariate time series prediction framework based on a transformer model consisting of convolutional neural networks (CNNs). The proposed model has a structure that extracts temporal features of input data through CNN and interprets correlations between variables through an attention mechanism. This framework solves the problem of the inability to simultaneously analyze the temporal features of the input data and the correlation between variables, which is a limitation of the forecasting models presented in existing studies. We designed a forecasting experiment using several time series datasets with various data characteristics to precisely evaluate the proposed model. In addition, comparative experiments were performed between the proposed model and several predictive models proposed in recent studies. Furthermore, we conducted ablation studies on the extent to which the proposed CNN structure in the prediction model affects the forecasting results by substituting a specific layer of the model. The results of the experiments showed that the proposed predictive model exhibited good performance in predicting time series data with a clear cycle and high correlation between variables, and improved the accuracy by approximately 3% to 5% compared with that of previous studies’ time series prediction models.https://ieeexplore.ieee.org/document/9874747/Artificial neural networkspredictive modelstime series prediction |
| spellingShingle | Dong-Keon Kim Kwangsu Kim A Convolutional Transformer Model for Multivariate Time Series Prediction IEEE Access Artificial neural networks predictive models time series prediction |
| title | A Convolutional Transformer Model for Multivariate Time Series Prediction |
| title_full | A Convolutional Transformer Model for Multivariate Time Series Prediction |
| title_fullStr | A Convolutional Transformer Model for Multivariate Time Series Prediction |
| title_full_unstemmed | A Convolutional Transformer Model for Multivariate Time Series Prediction |
| title_short | A Convolutional Transformer Model for Multivariate Time Series Prediction |
| title_sort | convolutional transformer model for multivariate time series prediction |
| topic | Artificial neural networks predictive models time series prediction |
| url | https://ieeexplore.ieee.org/document/9874747/ |
| work_keys_str_mv | AT dongkeonkim aconvolutionaltransformermodelformultivariatetimeseriesprediction AT kwangsukim aconvolutionaltransformermodelformultivariatetimeseriesprediction AT dongkeonkim convolutionaltransformermodelformultivariatetimeseriesprediction AT kwangsukim convolutionaltransformermodelformultivariatetimeseriesprediction |