A new stable and interpretable flood forecasting model combining multi-head attention mechanism and multiple linear regression
This article proposes a multi-head attention flood forecasting model (MHAFFM) that combines a multi-head attention mechanism (MHAM) with multiple linear regression for flood forecasting. Compared to models based on Long Short-Term Memory (LSTM) neural networks, MHAFFM enables precise and stable mult...
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IWA Publishing
2023-11-01
|
| Series: | Journal of Hydroinformatics |
| Subjects: | |
| Online Access: | http://jhydro.iwaponline.com/content/25/6/2561 |
| _version_ | 1827615091445465088 |
|---|---|
| author | Yi-yang Wang Wenchuan Wang Kwok-wing Chau Dong-mei Xu Hong-fei Zang Chang-jun Liu Qiang Ma |
| author_facet | Yi-yang Wang Wenchuan Wang Kwok-wing Chau Dong-mei Xu Hong-fei Zang Chang-jun Liu Qiang Ma |
| author_sort | Yi-yang Wang |
| collection | DOAJ |
| description | This article proposes a multi-head attention flood forecasting model (MHAFFM) that combines a multi-head attention mechanism (MHAM) with multiple linear regression for flood forecasting. Compared to models based on Long Short-Term Memory (LSTM) neural networks, MHAFFM enables precise and stable multi-hour flood forecasting. First, the model utilizes characteristics of full-batch stable input data in multiple linear regression to solve the problem of oscillation in the prediction results of existing models. Second, full-batch information is connected to MHAM to improve the model's ability to process and interpret high-dimensional information. Finally, the model accurately and stably predicts future flood processes through linear layers. The model is applied to Dawen River Basin, and experimental results show that the MHAFFM, compared to three benchmarking models, namely, LSTM, BOA-LSTM (LSTM with Bayesian Optimization Algorithm for Hyperparameter Tuning), and MHAM-LSTM (LSTM model with MHAM in hidden layer), significantly improves the prediction performance under different lead time scenarios while maintaining good stability and interpretability. Taking Nash–Sutcliffe efficiency index as an example, under a lead time of 3 h, the MHAFFM model exhibits improvements of 8.85, 3.71, and 10.29% compared to the three benchmarking models, respectively. This research provides a new approach for flood forecasting.
HIGHLIGHTS
Proposes a novel multi-head attention flood forecasting model (MHAFFM).;
Multi-head attention mechanism strengthens the model's ability to handle high-dimensional data.;
Linear layers effectively harness the performance of the multi-head attention mechanism.;
MHAFFM significantly enhances the stability of forecasted results.;
Even in longer lead time scenarios, the model maintains high accuracy and stability.; |
| first_indexed | 2024-03-09T09:05:14Z |
| format | Article |
| id | doaj.art-8fc12b807b0a4da4955c65b2e062197b |
| institution | Directory Open Access Journal |
| issn | 1464-7141 1465-1734 |
| language | English |
| last_indexed | 2024-03-09T09:05:14Z |
| publishDate | 2023-11-01 |
| publisher | IWA Publishing |
| record_format | Article |
| series | Journal of Hydroinformatics |
| spelling | doaj.art-8fc12b807b0a4da4955c65b2e062197b2023-12-02T10:28:09ZengIWA PublishingJournal of Hydroinformatics1464-71411465-17342023-11-012562561258810.2166/hydro.2023.160160A new stable and interpretable flood forecasting model combining multi-head attention mechanism and multiple linear regressionYi-yang Wang0Wenchuan Wang1Kwok-wing Chau2Dong-mei Xu3Hong-fei Zang4Chang-jun Liu5Qiang Ma6 College of Water Resources, North China University of Water Resources and Electric Power, Zhengzhou 450046, China College of Water Resources, North China University of Water Resources and Electric Power, Zhengzhou 450046, China Department of Civil and Environmental Engineering, The Hong Kong Polytechnic University, Hung Hom, Kowloon, Hong Kong, China College of Water Resources, North China University of Water Resources and Electric Power, Zhengzhou 450046, China College of Water Resources, North China University of Water Resources and Electric Power, Zhengzhou 450046, China Research Center on Flood and Drought Disaster Reduction, China Institute of Water Resources and Hydropower Research, Beijing 100081, China Research Center on Flood and Drought Disaster Reduction, China Institute of Water Resources and Hydropower Research, Beijing 100081, China This article proposes a multi-head attention flood forecasting model (MHAFFM) that combines a multi-head attention mechanism (MHAM) with multiple linear regression for flood forecasting. Compared to models based on Long Short-Term Memory (LSTM) neural networks, MHAFFM enables precise and stable multi-hour flood forecasting. First, the model utilizes characteristics of full-batch stable input data in multiple linear regression to solve the problem of oscillation in the prediction results of existing models. Second, full-batch information is connected to MHAM to improve the model's ability to process and interpret high-dimensional information. Finally, the model accurately and stably predicts future flood processes through linear layers. The model is applied to Dawen River Basin, and experimental results show that the MHAFFM, compared to three benchmarking models, namely, LSTM, BOA-LSTM (LSTM with Bayesian Optimization Algorithm for Hyperparameter Tuning), and MHAM-LSTM (LSTM model with MHAM in hidden layer), significantly improves the prediction performance under different lead time scenarios while maintaining good stability and interpretability. Taking Nash–Sutcliffe efficiency index as an example, under a lead time of 3 h, the MHAFFM model exhibits improvements of 8.85, 3.71, and 10.29% compared to the three benchmarking models, respectively. This research provides a new approach for flood forecasting. HIGHLIGHTS Proposes a novel multi-head attention flood forecasting model (MHAFFM).; Multi-head attention mechanism strengthens the model's ability to handle high-dimensional data.; Linear layers effectively harness the performance of the multi-head attention mechanism.; MHAFFM significantly enhances the stability of forecasted results.; Even in longer lead time scenarios, the model maintains high accuracy and stability.;http://jhydro.iwaponline.com/content/25/6/2561flood forecastinglstmmulti-head attention mechanismmultiple linear regression |
| spellingShingle | Yi-yang Wang Wenchuan Wang Kwok-wing Chau Dong-mei Xu Hong-fei Zang Chang-jun Liu Qiang Ma A new stable and interpretable flood forecasting model combining multi-head attention mechanism and multiple linear regression Journal of Hydroinformatics flood forecasting lstm multi-head attention mechanism multiple linear regression |
| title | A new stable and interpretable flood forecasting model combining multi-head attention mechanism and multiple linear regression |
| title_full | A new stable and interpretable flood forecasting model combining multi-head attention mechanism and multiple linear regression |
| title_fullStr | A new stable and interpretable flood forecasting model combining multi-head attention mechanism and multiple linear regression |
| title_full_unstemmed | A new stable and interpretable flood forecasting model combining multi-head attention mechanism and multiple linear regression |
| title_short | A new stable and interpretable flood forecasting model combining multi-head attention mechanism and multiple linear regression |
| title_sort | new stable and interpretable flood forecasting model combining multi head attention mechanism and multiple linear regression |
| topic | flood forecasting lstm multi-head attention mechanism multiple linear regression |
| url | http://jhydro.iwaponline.com/content/25/6/2561 |
| work_keys_str_mv | AT yiyangwang anewstableandinterpretablefloodforecastingmodelcombiningmultiheadattentionmechanismandmultiplelinearregression AT wenchuanwang anewstableandinterpretablefloodforecastingmodelcombiningmultiheadattentionmechanismandmultiplelinearregression AT kwokwingchau anewstableandinterpretablefloodforecastingmodelcombiningmultiheadattentionmechanismandmultiplelinearregression AT dongmeixu anewstableandinterpretablefloodforecastingmodelcombiningmultiheadattentionmechanismandmultiplelinearregression AT hongfeizang anewstableandinterpretablefloodforecastingmodelcombiningmultiheadattentionmechanismandmultiplelinearregression AT changjunliu anewstableandinterpretablefloodforecastingmodelcombiningmultiheadattentionmechanismandmultiplelinearregression AT qiangma anewstableandinterpretablefloodforecastingmodelcombiningmultiheadattentionmechanismandmultiplelinearregression AT yiyangwang newstableandinterpretablefloodforecastingmodelcombiningmultiheadattentionmechanismandmultiplelinearregression AT wenchuanwang newstableandinterpretablefloodforecastingmodelcombiningmultiheadattentionmechanismandmultiplelinearregression AT kwokwingchau newstableandinterpretablefloodforecastingmodelcombiningmultiheadattentionmechanismandmultiplelinearregression AT dongmeixu newstableandinterpretablefloodforecastingmodelcombiningmultiheadattentionmechanismandmultiplelinearregression AT hongfeizang newstableandinterpretablefloodforecastingmodelcombiningmultiheadattentionmechanismandmultiplelinearregression AT changjunliu newstableandinterpretablefloodforecastingmodelcombiningmultiheadattentionmechanismandmultiplelinearregression AT qiangma newstableandinterpretablefloodforecastingmodelcombiningmultiheadattentionmechanismandmultiplelinearregression |