Transformer training strategies for forecasting multiple load time series
Abstract In the smart grid of the future, accurate load forecasts on the level of individual clients can help to balance supply and demand locally and to prevent grid outages. While the number of monitored clients will increase with the ongoing smart meter rollout, the amount of data per client will...
Main Authors: | , , , , , , |
---|---|
Format: | Article |
Language: | English |
Published: |
SpringerOpen
2023-10-01
|
Series: | Energy Informatics |
Subjects: | |
Online Access: | https://doi.org/10.1186/s42162-023-00278-z |
_version_ | 1797653570775941120 |
---|---|
author | Matthias Hertel Maximilian Beichter Benedikt Heidrich Oliver Neumann Benjamin Schäfer Ralf Mikut Veit Hagenmeyer |
author_facet | Matthias Hertel Maximilian Beichter Benedikt Heidrich Oliver Neumann Benjamin Schäfer Ralf Mikut Veit Hagenmeyer |
author_sort | Matthias Hertel |
collection | DOAJ |
description | Abstract In the smart grid of the future, accurate load forecasts on the level of individual clients can help to balance supply and demand locally and to prevent grid outages. While the number of monitored clients will increase with the ongoing smart meter rollout, the amount of data per client will always be limited. We evaluate whether a Transformer load forecasting model benefits from a transfer learning strategy, where a global univariate model is trained on the load time series from multiple clients. In experiments with two datasets containing load time series from several hundred clients, we find that the global training strategy is superior to the multivariate and local training strategies used in related work. On average, the global training strategy results in 21.8% and 12.8% lower forecasting errors than the two other strategies, measured across forecasting horizons from one day to one month into the future. A comparison to linear models, multi-layer perceptrons and LSTMs shows that Transformers are effective for load forecasting when they are trained with the global training strategy. |
first_indexed | 2024-03-11T16:46:30Z |
format | Article |
id | doaj.art-ac1d3b055e1641a6aa82f0c56bbac8fa |
institution | Directory Open Access Journal |
issn | 2520-8942 |
language | English |
last_indexed | 2024-03-11T16:46:30Z |
publishDate | 2023-10-01 |
publisher | SpringerOpen |
record_format | Article |
series | Energy Informatics |
spelling | doaj.art-ac1d3b055e1641a6aa82f0c56bbac8fa2023-10-22T11:28:54ZengSpringerOpenEnergy Informatics2520-89422023-10-016S111310.1186/s42162-023-00278-zTransformer training strategies for forecasting multiple load time seriesMatthias Hertel0Maximilian Beichter1Benedikt Heidrich2Oliver Neumann3Benjamin Schäfer4Ralf Mikut5Veit Hagenmeyer6Karlsruhe Institute of Technology, Institute for Automation and Applied InformaticsKarlsruhe Institute of Technology, Institute for Automation and Applied InformaticsKarlsruhe Institute of Technology, Institute for Automation and Applied InformaticsKarlsruhe Institute of Technology, Institute for Automation and Applied InformaticsKarlsruhe Institute of Technology, Institute for Automation and Applied InformaticsKarlsruhe Institute of Technology, Institute for Automation and Applied InformaticsKarlsruhe Institute of Technology, Institute for Automation and Applied InformaticsAbstract In the smart grid of the future, accurate load forecasts on the level of individual clients can help to balance supply and demand locally and to prevent grid outages. While the number of monitored clients will increase with the ongoing smart meter rollout, the amount of data per client will always be limited. We evaluate whether a Transformer load forecasting model benefits from a transfer learning strategy, where a global univariate model is trained on the load time series from multiple clients. In experiments with two datasets containing load time series from several hundred clients, we find that the global training strategy is superior to the multivariate and local training strategies used in related work. On average, the global training strategy results in 21.8% and 12.8% lower forecasting errors than the two other strategies, measured across forecasting horizons from one day to one month into the future. A comparison to linear models, multi-layer perceptrons and LSTMs shows that Transformers are effective for load forecasting when they are trained with the global training strategy.https://doi.org/10.1186/s42162-023-00278-zLoad forecastingTransformerGlobal modelTime seriesSmart grid |
spellingShingle | Matthias Hertel Maximilian Beichter Benedikt Heidrich Oliver Neumann Benjamin Schäfer Ralf Mikut Veit Hagenmeyer Transformer training strategies for forecasting multiple load time series Energy Informatics Load forecasting Transformer Global model Time series Smart grid |
title | Transformer training strategies for forecasting multiple load time series |
title_full | Transformer training strategies for forecasting multiple load time series |
title_fullStr | Transformer training strategies for forecasting multiple load time series |
title_full_unstemmed | Transformer training strategies for forecasting multiple load time series |
title_short | Transformer training strategies for forecasting multiple load time series |
title_sort | transformer training strategies for forecasting multiple load time series |
topic | Load forecasting Transformer Global model Time series Smart grid |
url | https://doi.org/10.1186/s42162-023-00278-z |
work_keys_str_mv | AT matthiashertel transformertrainingstrategiesforforecastingmultipleloadtimeseries AT maximilianbeichter transformertrainingstrategiesforforecastingmultipleloadtimeseries AT benediktheidrich transformertrainingstrategiesforforecastingmultipleloadtimeseries AT oliverneumann transformertrainingstrategiesforforecastingmultipleloadtimeseries AT benjaminschafer transformertrainingstrategiesforforecastingmultipleloadtimeseries AT ralfmikut transformertrainingstrategiesforforecastingmultipleloadtimeseries AT veithagenmeyer transformertrainingstrategiesforforecastingmultipleloadtimeseries |