An adaptive deep-learning load forecasting framework by integrating transformer and domain knowledge
Electrical energy is essential in today's society. Accurate electrical load forecasting is beneficial for better scheduling of electricity generation and saving electrical energy. In this paper, we propose an adaptive deep-learning load forecasting framework by integrating Transformer and domai...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Elsevier
2023-06-01
|
| Series: | Advances in Applied Energy |
| Subjects: | |
| Online Access: | http://www.sciencedirect.com/science/article/pii/S2666792423000215 |
| _version_ | 1797807189990047744 |
|---|---|
| author | Jiaxin Gao Yuntian Chen Wenbo Hu Dongxiao Zhang |
| author_facet | Jiaxin Gao Yuntian Chen Wenbo Hu Dongxiao Zhang |
| author_sort | Jiaxin Gao |
| collection | DOAJ |
| description | Electrical energy is essential in today's society. Accurate electrical load forecasting is beneficial for better scheduling of electricity generation and saving electrical energy. In this paper, we propose an adaptive deep-learning load forecasting framework by integrating Transformer and domain knowledge (Adaptive-TgDLF). Adaptive-TgDLF introduces the deep-learning model Transformer and adaptive learning methods (including transfer learning for different locations and online learning for different time periods), which captures the long-term dependency of the load series, and is more appropriate for realistic scenarios with scarce samples and variable data distributions. Under the theory-guided framework, the electrical load is divided into dimensionless trends and local fluctuations. The dimensionless trends are considered as the inherent pattern of the load, and the local fluctuations are considered to be determined by the external driving forces. Adaptive learning can cope with the change of load in location and time, and can make full use of load data at different locations and times to train a more efficient model. Cross-validation experiments on different districts show that Adaptive-TgDLF is approximately 16% more accurate than the previous TgDLF model and saves more than half of the training time. Adaptive-TgDLF with 50% weather noise has the same accuracy as the previous TgDLF model without noise, which proves its robustness. We also preliminarily mine the interpretability of Transformer in Adaptive-TgDLF, which may provide future potential for better theory guidance. Furthermore, experiments demonstrate that transfer learning can accelerate convergence of the model in half the number of training epochs and achieve better performance, and online learning enables the model to achieve better results on the changing load. |
| first_indexed | 2024-03-13T06:18:50Z |
| format | Article |
| id | doaj.art-991d436c3d124e78966d5fa58ffa2d2f |
| institution | Directory Open Access Journal |
| issn | 2666-7924 |
| language | English |
| last_indexed | 2024-03-13T06:18:50Z |
| publishDate | 2023-06-01 |
| publisher | Elsevier |
| record_format | Article |
| series | Advances in Applied Energy |
| spelling | doaj.art-991d436c3d124e78966d5fa58ffa2d2f2023-06-10T04:28:47ZengElsevierAdvances in Applied Energy2666-79242023-06-0110100142An adaptive deep-learning load forecasting framework by integrating transformer and domain knowledgeJiaxin Gao0Yuntian Chen1Wenbo Hu2Dongxiao Zhang3Eastern Institute for Advanced Study, Eastern Institute of Technology, Ningbo, Zhejiang, P. R. China; School of Electronic Information and Electrical Engineering, Shanghai Jiaotong University, Shanghai, P. R. ChinaEastern Institute for Advanced Study, Eastern Institute of Technology, Ningbo, Zhejiang, P. R. China; School of Electronic Information and Electrical Engineering, Shanghai Jiaotong University, Shanghai, P. R. China; Corresponding authors.School of Computer and Information, Hefei University of Technology, Hefei, P. R. ChinaEastern Institute for Advanced Study, Eastern Institute of Technology, Ningbo, Zhejiang, P. R. China; Department of Mathematics and Theories, Peng Cheng Laboratory, Guangdong, P. R. China; National Center for Applied Mathematics Shenzhen (NCAMS), Southern University of Science and Technology, Guangdong, P. R. China; Corresponding authors.Electrical energy is essential in today's society. Accurate electrical load forecasting is beneficial for better scheduling of electricity generation and saving electrical energy. In this paper, we propose an adaptive deep-learning load forecasting framework by integrating Transformer and domain knowledge (Adaptive-TgDLF). Adaptive-TgDLF introduces the deep-learning model Transformer and adaptive learning methods (including transfer learning for different locations and online learning for different time periods), which captures the long-term dependency of the load series, and is more appropriate for realistic scenarios with scarce samples and variable data distributions. Under the theory-guided framework, the electrical load is divided into dimensionless trends and local fluctuations. The dimensionless trends are considered as the inherent pattern of the load, and the local fluctuations are considered to be determined by the external driving forces. Adaptive learning can cope with the change of load in location and time, and can make full use of load data at different locations and times to train a more efficient model. Cross-validation experiments on different districts show that Adaptive-TgDLF is approximately 16% more accurate than the previous TgDLF model and saves more than half of the training time. Adaptive-TgDLF with 50% weather noise has the same accuracy as the previous TgDLF model without noise, which proves its robustness. We also preliminarily mine the interpretability of Transformer in Adaptive-TgDLF, which may provide future potential for better theory guidance. Furthermore, experiments demonstrate that transfer learning can accelerate convergence of the model in half the number of training epochs and achieve better performance, and online learning enables the model to achieve better results on the changing load.http://www.sciencedirect.com/science/article/pii/S2666792423000215Load forecastingDeep-learningDomain knowledgeTransfer learningOnline learningInterpretability |
| spellingShingle | Jiaxin Gao Yuntian Chen Wenbo Hu Dongxiao Zhang An adaptive deep-learning load forecasting framework by integrating transformer and domain knowledge Advances in Applied Energy Load forecasting Deep-learning Domain knowledge Transfer learning Online learning Interpretability |
| title | An adaptive deep-learning load forecasting framework by integrating transformer and domain knowledge |
| title_full | An adaptive deep-learning load forecasting framework by integrating transformer and domain knowledge |
| title_fullStr | An adaptive deep-learning load forecasting framework by integrating transformer and domain knowledge |
| title_full_unstemmed | An adaptive deep-learning load forecasting framework by integrating transformer and domain knowledge |
| title_short | An adaptive deep-learning load forecasting framework by integrating transformer and domain knowledge |
| title_sort | adaptive deep learning load forecasting framework by integrating transformer and domain knowledge |
| topic | Load forecasting Deep-learning Domain knowledge Transfer learning Online learning Interpretability |
| url | http://www.sciencedirect.com/science/article/pii/S2666792423000215 |
| work_keys_str_mv | AT jiaxingao anadaptivedeeplearningloadforecastingframeworkbyintegratingtransformeranddomainknowledge AT yuntianchen anadaptivedeeplearningloadforecastingframeworkbyintegratingtransformeranddomainknowledge AT wenbohu anadaptivedeeplearningloadforecastingframeworkbyintegratingtransformeranddomainknowledge AT dongxiaozhang anadaptivedeeplearningloadforecastingframeworkbyintegratingtransformeranddomainknowledge AT jiaxingao adaptivedeeplearningloadforecastingframeworkbyintegratingtransformeranddomainknowledge AT yuntianchen adaptivedeeplearningloadforecastingframeworkbyintegratingtransformeranddomainknowledge AT wenbohu adaptivedeeplearningloadforecastingframeworkbyintegratingtransformeranddomainknowledge AT dongxiaozhang adaptivedeeplearningloadforecastingframeworkbyintegratingtransformeranddomainknowledge |