Multi-Task Representation Learning for Renewable-Power Forecasting: A Comparative Analysis of Unified Autoencoder Variants and Task-Embedding Dimensions
Typically, renewable-power-generation forecasting using machine learning involves creating separate models for each photovoltaic or wind park, known as single-task learning models. However, transfer learning has gained popularity in recent years, as it allows for the transfer of knowledge from sourc...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-09-01
|
| Series: | Machine Learning and Knowledge Extraction |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2504-4990/5/3/62 |
| _version_ | 1797579074667806720 |
|---|---|
| author | Chandana Priya Nivarthi Stephan Vogt Bernhard Sick |
| author_facet | Chandana Priya Nivarthi Stephan Vogt Bernhard Sick |
| author_sort | Chandana Priya Nivarthi |
| collection | DOAJ |
| description | Typically, renewable-power-generation forecasting using machine learning involves creating separate models for each photovoltaic or wind park, known as single-task learning models. However, transfer learning has gained popularity in recent years, as it allows for the transfer of knowledge from source parks to target parks. Nevertheless, determining the most similar source park(s) for transfer learning can be challenging, particularly when the target park has limited or no historical data samples. To address this issue, we propose a multi-task learning architecture that employs a Unified Autoencoder (UAE) to initially learn a common representation of input weather features among tasks and then utilizes a Task-Embedding layer in a Neural Network (TENN) to learn task-specific information. This proposed UAE-TENN architecture can be easily extended to new parks with or without historical data. We evaluate the performance of our proposed architecture and compare it to single-task learning models on six photovoltaic and wind farm datasets consisting of a total of 529 parks. Our results show that the UAE-TENN architecture significantly improves power-forecasting performance by 10 to 19% for photovoltaic parks and 5 to 15% for wind parks compared to baseline models. We also demonstrate that UAE-TENN improves forecast accuracy for a new park by 19% for photovoltaic parks, even in a zero-shot learning scenario where there is no historical data. Additionally, we propose variants of the Unified Autoencoder with convolutional and LSTM layers, compare their performance, and provide a comparison among architectures with different numbers of task-embedding dimensions. Finally, we demonstrate the utility of trained task embeddings for interpretation and visualization purposes. |
| first_indexed | 2024-03-10T22:31:48Z |
| format | Article |
| id | doaj.art-2ff5e1197e584b16af58f9feeebf5929 |
| institution | Directory Open Access Journal |
| issn | 2504-4990 |
| language | English |
| last_indexed | 2024-03-10T22:31:48Z |
| publishDate | 2023-09-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Machine Learning and Knowledge Extraction |
| spelling | doaj.art-2ff5e1197e584b16af58f9feeebf59292023-11-19T11:41:52ZengMDPI AGMachine Learning and Knowledge Extraction2504-49902023-09-01531214123310.3390/make5030062Multi-Task Representation Learning for Renewable-Power Forecasting: A Comparative Analysis of Unified Autoencoder Variants and Task-Embedding DimensionsChandana Priya Nivarthi0Stephan Vogt1Bernhard Sick2Intelligent Embedded Systems, University of Kassel, Wilhelmshöher Allee 73, 34121 Kassel, GermanyIntelligent Embedded Systems, University of Kassel, Wilhelmshöher Allee 73, 34121 Kassel, GermanyIntelligent Embedded Systems, University of Kassel, Wilhelmshöher Allee 73, 34121 Kassel, GermanyTypically, renewable-power-generation forecasting using machine learning involves creating separate models for each photovoltaic or wind park, known as single-task learning models. However, transfer learning has gained popularity in recent years, as it allows for the transfer of knowledge from source parks to target parks. Nevertheless, determining the most similar source park(s) for transfer learning can be challenging, particularly when the target park has limited or no historical data samples. To address this issue, we propose a multi-task learning architecture that employs a Unified Autoencoder (UAE) to initially learn a common representation of input weather features among tasks and then utilizes a Task-Embedding layer in a Neural Network (TENN) to learn task-specific information. This proposed UAE-TENN architecture can be easily extended to new parks with or without historical data. We evaluate the performance of our proposed architecture and compare it to single-task learning models on six photovoltaic and wind farm datasets consisting of a total of 529 parks. Our results show that the UAE-TENN architecture significantly improves power-forecasting performance by 10 to 19% for photovoltaic parks and 5 to 15% for wind parks compared to baseline models. We also demonstrate that UAE-TENN improves forecast accuracy for a new park by 19% for photovoltaic parks, even in a zero-shot learning scenario where there is no historical data. Additionally, we propose variants of the Unified Autoencoder with convolutional and LSTM layers, compare their performance, and provide a comparison among architectures with different numbers of task-embedding dimensions. Finally, we demonstrate the utility of trained task embeddings for interpretation and visualization purposes.https://www.mdpi.com/2504-4990/5/3/62transfer learningmulti-task learningzero-shot learningautoencoderspower forecast |
| spellingShingle | Chandana Priya Nivarthi Stephan Vogt Bernhard Sick Multi-Task Representation Learning for Renewable-Power Forecasting: A Comparative Analysis of Unified Autoencoder Variants and Task-Embedding Dimensions Machine Learning and Knowledge Extraction transfer learning multi-task learning zero-shot learning autoencoders power forecast |
| title | Multi-Task Representation Learning for Renewable-Power Forecasting: A Comparative Analysis of Unified Autoencoder Variants and Task-Embedding Dimensions |
| title_full | Multi-Task Representation Learning for Renewable-Power Forecasting: A Comparative Analysis of Unified Autoencoder Variants and Task-Embedding Dimensions |
| title_fullStr | Multi-Task Representation Learning for Renewable-Power Forecasting: A Comparative Analysis of Unified Autoencoder Variants and Task-Embedding Dimensions |
| title_full_unstemmed | Multi-Task Representation Learning for Renewable-Power Forecasting: A Comparative Analysis of Unified Autoencoder Variants and Task-Embedding Dimensions |
| title_short | Multi-Task Representation Learning for Renewable-Power Forecasting: A Comparative Analysis of Unified Autoencoder Variants and Task-Embedding Dimensions |
| title_sort | multi task representation learning for renewable power forecasting a comparative analysis of unified autoencoder variants and task embedding dimensions |
| topic | transfer learning multi-task learning zero-shot learning autoencoders power forecast |
| url | https://www.mdpi.com/2504-4990/5/3/62 |
| work_keys_str_mv | AT chandanapriyanivarthi multitaskrepresentationlearningforrenewablepowerforecastingacomparativeanalysisofunifiedautoencodervariantsandtaskembeddingdimensions AT stephanvogt multitaskrepresentationlearningforrenewablepowerforecastingacomparativeanalysisofunifiedautoencodervariantsandtaskembeddingdimensions AT bernhardsick multitaskrepresentationlearningforrenewablepowerforecastingacomparativeanalysisofunifiedautoencodervariantsandtaskembeddingdimensions |