Transformer training strategies for forecasting multiple load time series

Abstract In the smart grid of the future, accurate load forecasts on the level of individual clients can help to balance supply and demand locally and to prevent grid outages. While the number of monitored clients will increase with the ongoing smart meter rollout, the amount of data per client will...

Full description

Bibliographic Details
Main Authors: Matthias Hertel, Maximilian Beichter, Benedikt Heidrich, Oliver Neumann, Benjamin Schäfer, Ralf Mikut, Veit Hagenmeyer
Format: Article
Language:English
Published: SpringerOpen 2023-10-01
Series:Energy Informatics
Subjects:
Online Access:https://doi.org/10.1186/s42162-023-00278-z
_version_ 1797653570775941120
author Matthias Hertel
Maximilian Beichter
Benedikt Heidrich
Oliver Neumann
Benjamin Schäfer
Ralf Mikut
Veit Hagenmeyer
author_facet Matthias Hertel
Maximilian Beichter
Benedikt Heidrich
Oliver Neumann
Benjamin Schäfer
Ralf Mikut
Veit Hagenmeyer
author_sort Matthias Hertel
collection DOAJ
description Abstract In the smart grid of the future, accurate load forecasts on the level of individual clients can help to balance supply and demand locally and to prevent grid outages. While the number of monitored clients will increase with the ongoing smart meter rollout, the amount of data per client will always be limited. We evaluate whether a Transformer load forecasting model benefits from a transfer learning strategy, where a global univariate model is trained on the load time series from multiple clients. In experiments with two datasets containing load time series from several hundred clients, we find that the global training strategy is superior to the multivariate and local training strategies used in related work. On average, the global training strategy results in 21.8% and 12.8% lower forecasting errors than the two other strategies, measured across forecasting horizons from one day to one month into the future. A comparison to linear models, multi-layer perceptrons and LSTMs shows that Transformers are effective for load forecasting when they are trained with the global training strategy.
first_indexed 2024-03-11T16:46:30Z
format Article
id doaj.art-ac1d3b055e1641a6aa82f0c56bbac8fa
institution Directory Open Access Journal
issn 2520-8942
language English
last_indexed 2024-03-11T16:46:30Z
publishDate 2023-10-01
publisher SpringerOpen
record_format Article
series Energy Informatics
spelling doaj.art-ac1d3b055e1641a6aa82f0c56bbac8fa2023-10-22T11:28:54ZengSpringerOpenEnergy Informatics2520-89422023-10-016S111310.1186/s42162-023-00278-zTransformer training strategies for forecasting multiple load time seriesMatthias Hertel0Maximilian Beichter1Benedikt Heidrich2Oliver Neumann3Benjamin Schäfer4Ralf Mikut5Veit Hagenmeyer6Karlsruhe Institute of Technology, Institute for Automation and Applied InformaticsKarlsruhe Institute of Technology, Institute for Automation and Applied InformaticsKarlsruhe Institute of Technology, Institute for Automation and Applied InformaticsKarlsruhe Institute of Technology, Institute for Automation and Applied InformaticsKarlsruhe Institute of Technology, Institute for Automation and Applied InformaticsKarlsruhe Institute of Technology, Institute for Automation and Applied InformaticsKarlsruhe Institute of Technology, Institute for Automation and Applied InformaticsAbstract In the smart grid of the future, accurate load forecasts on the level of individual clients can help to balance supply and demand locally and to prevent grid outages. While the number of monitored clients will increase with the ongoing smart meter rollout, the amount of data per client will always be limited. We evaluate whether a Transformer load forecasting model benefits from a transfer learning strategy, where a global univariate model is trained on the load time series from multiple clients. In experiments with two datasets containing load time series from several hundred clients, we find that the global training strategy is superior to the multivariate and local training strategies used in related work. On average, the global training strategy results in 21.8% and 12.8% lower forecasting errors than the two other strategies, measured across forecasting horizons from one day to one month into the future. A comparison to linear models, multi-layer perceptrons and LSTMs shows that Transformers are effective for load forecasting when they are trained with the global training strategy.https://doi.org/10.1186/s42162-023-00278-zLoad forecastingTransformerGlobal modelTime seriesSmart grid
spellingShingle Matthias Hertel
Maximilian Beichter
Benedikt Heidrich
Oliver Neumann
Benjamin Schäfer
Ralf Mikut
Veit Hagenmeyer
Transformer training strategies for forecasting multiple load time series
Energy Informatics
Load forecasting
Transformer
Global model
Time series
Smart grid
title Transformer training strategies for forecasting multiple load time series
title_full Transformer training strategies for forecasting multiple load time series
title_fullStr Transformer training strategies for forecasting multiple load time series
title_full_unstemmed Transformer training strategies for forecasting multiple load time series
title_short Transformer training strategies for forecasting multiple load time series
title_sort transformer training strategies for forecasting multiple load time series
topic Load forecasting
Transformer
Global model
Time series
Smart grid
url https://doi.org/10.1186/s42162-023-00278-z
work_keys_str_mv AT matthiashertel transformertrainingstrategiesforforecastingmultipleloadtimeseries
AT maximilianbeichter transformertrainingstrategiesforforecastingmultipleloadtimeseries
AT benediktheidrich transformertrainingstrategiesforforecastingmultipleloadtimeseries
AT oliverneumann transformertrainingstrategiesforforecastingmultipleloadtimeseries
AT benjaminschafer transformertrainingstrategiesforforecastingmultipleloadtimeseries
AT ralfmikut transformertrainingstrategiesforforecastingmultipleloadtimeseries
AT veithagenmeyer transformertrainingstrategiesforforecastingmultipleloadtimeseries