Hyperparameter Optimization of Bayesian Neural Network Using Bayesian Optimization and Intelligent Feature Engineering for Load Forecasting
This paper proposes a new hybrid framework for short-term load forecasting (STLF) by combining the Feature Engineering (FE) and Bayesian Optimization (BO) algorithms with a Bayesian Neural Network (BNN). The FE module comprises feature selection and extraction phases. Firstly, by merging the Random...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2022-06-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/22/12/4446 |
| _version_ | 1797482481921818624 |
|---|---|
| author | M. Zulfiqar Kelum A. A. Gamage M. Kamran M. B. Rasheed |
| author_facet | M. Zulfiqar Kelum A. A. Gamage M. Kamran M. B. Rasheed |
| author_sort | M. Zulfiqar |
| collection | DOAJ |
| description | This paper proposes a new hybrid framework for short-term load forecasting (STLF) by combining the Feature Engineering (FE) and Bayesian Optimization (BO) algorithms with a Bayesian Neural Network (BNN). The FE module comprises feature selection and extraction phases. Firstly, by merging the Random Forest (RaF) and Relief-F (ReF) algorithms, we developed a hybrid feature selector based on grey correlation analysis (GCA) to eliminate feature redundancy. Secondly, a radial basis Kernel function and principal component analysis (KPCA) are integrated into the feature-extraction module for dimensional reduction. Thirdly, the Bayesian Optimization (BO) algorithm is used to fine-tune the control parameters of a BNN and provides more accurate results by avoiding the optimal local trapping. The proposed FE-BNN-BO framework works in such a way to ensure stability, convergence, and accuracy. The proposed FE-BNN-BO model is tested on the hourly load data obtained from the PJM, USA, electricity market. In addition, the simulation results are also compared with other benchmark models such as Bi-Level, long short-term memory (LSTM), an accurate and fast convergence-based ANN (ANN-AFC), and a mutual-information-based ANN (ANN-MI). The results show that the proposed model has significantly improved the accuracy with a fast convergence rate and reduced the mean absolute percent error (MAPE). |
| first_indexed | 2024-03-09T22:33:00Z |
| format | Article |
| id | doaj.art-888a26ee73b24aa2b5d487e79fd5412e |
| institution | Directory Open Access Journal |
| issn | 1424-8220 |
| language | English |
| last_indexed | 2024-03-09T22:33:00Z |
| publishDate | 2022-06-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj.art-888a26ee73b24aa2b5d487e79fd5412e2023-11-23T18:53:34ZengMDPI AGSensors1424-82202022-06-012212444610.3390/s22124446Hyperparameter Optimization of Bayesian Neural Network Using Bayesian Optimization and Intelligent Feature Engineering for Load ForecastingM. Zulfiqar0Kelum A. A. Gamage1M. Kamran2M. B. Rasheed3Department of Telecommunication Systems, Bahauddin Zakariya University, Multan 60000, PakistanJames Watt School of Engineering, James Watt South Building, University of Glasgow, Glasgow G12 8QQ, UKDepartment of Electrical Engineering, University of Engineering and Technology, Lahore 54000, PakistanEscuela Politécnica Superior, Universidad de Alcalá, ISG, 28805 Alcalá de Henares, SpainThis paper proposes a new hybrid framework for short-term load forecasting (STLF) by combining the Feature Engineering (FE) and Bayesian Optimization (BO) algorithms with a Bayesian Neural Network (BNN). The FE module comprises feature selection and extraction phases. Firstly, by merging the Random Forest (RaF) and Relief-F (ReF) algorithms, we developed a hybrid feature selector based on grey correlation analysis (GCA) to eliminate feature redundancy. Secondly, a radial basis Kernel function and principal component analysis (KPCA) are integrated into the feature-extraction module for dimensional reduction. Thirdly, the Bayesian Optimization (BO) algorithm is used to fine-tune the control parameters of a BNN and provides more accurate results by avoiding the optimal local trapping. The proposed FE-BNN-BO framework works in such a way to ensure stability, convergence, and accuracy. The proposed FE-BNN-BO model is tested on the hourly load data obtained from the PJM, USA, electricity market. In addition, the simulation results are also compared with other benchmark models such as Bi-Level, long short-term memory (LSTM), an accurate and fast convergence-based ANN (ANN-AFC), and a mutual-information-based ANN (ANN-MI). The results show that the proposed model has significantly improved the accuracy with a fast convergence rate and reduced the mean absolute percent error (MAPE).https://www.mdpi.com/1424-8220/22/12/4446Bayesian Neural NetworksBayesian Optimizationconvergence rateHamilton dynamicelectric load forecasting |
| spellingShingle | M. Zulfiqar Kelum A. A. Gamage M. Kamran M. B. Rasheed Hyperparameter Optimization of Bayesian Neural Network Using Bayesian Optimization and Intelligent Feature Engineering for Load Forecasting Sensors Bayesian Neural Networks Bayesian Optimization convergence rate Hamilton dynamic electric load forecasting |
| title | Hyperparameter Optimization of Bayesian Neural Network Using Bayesian Optimization and Intelligent Feature Engineering for Load Forecasting |
| title_full | Hyperparameter Optimization of Bayesian Neural Network Using Bayesian Optimization and Intelligent Feature Engineering for Load Forecasting |
| title_fullStr | Hyperparameter Optimization of Bayesian Neural Network Using Bayesian Optimization and Intelligent Feature Engineering for Load Forecasting |
| title_full_unstemmed | Hyperparameter Optimization of Bayesian Neural Network Using Bayesian Optimization and Intelligent Feature Engineering for Load Forecasting |
| title_short | Hyperparameter Optimization of Bayesian Neural Network Using Bayesian Optimization and Intelligent Feature Engineering for Load Forecasting |
| title_sort | hyperparameter optimization of bayesian neural network using bayesian optimization and intelligent feature engineering for load forecasting |
| topic | Bayesian Neural Networks Bayesian Optimization convergence rate Hamilton dynamic electric load forecasting |
| url | https://www.mdpi.com/1424-8220/22/12/4446 |
| work_keys_str_mv | AT mzulfiqar hyperparameteroptimizationofbayesianneuralnetworkusingbayesianoptimizationandintelligentfeatureengineeringforloadforecasting AT kelumaagamage hyperparameteroptimizationofbayesianneuralnetworkusingbayesianoptimizationandintelligentfeatureengineeringforloadforecasting AT mkamran hyperparameteroptimizationofbayesianneuralnetworkusingbayesianoptimizationandintelligentfeatureengineeringforloadforecasting AT mbrasheed hyperparameteroptimizationofbayesianneuralnetworkusingbayesianoptimizationandintelligentfeatureengineeringforloadforecasting |