Assessing Objective Functions in Streamflow Prediction Model Training Based on the Naïve Method
Reliable streamflow forecasting is a determining factor for water resource planning and flood control. To better understand the strengths and weaknesses of newly proposed methods in streamflow forecasting and facilitate comparisons of different research results, we test a simple, universal, and effi...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2024-03-01
|
| Series: | Water |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2073-4441/16/5/777 |
| _version_ | 1797263798773481472 |
|---|---|
| author | Yongen Lin Dagang Wang Tao Jiang Aiqing Kang |
| author_facet | Yongen Lin Dagang Wang Tao Jiang Aiqing Kang |
| author_sort | Yongen Lin |
| collection | DOAJ |
| description | Reliable streamflow forecasting is a determining factor for water resource planning and flood control. To better understand the strengths and weaknesses of newly proposed methods in streamflow forecasting and facilitate comparisons of different research results, we test a simple, universal, and efficient benchmark method, namely, the naïve method, for short-term streamflow prediction. Using the naïve method, we assess the streamflow forecasting performance of the long short-term memory models trained with different objective functions, including mean squared error (MSE), root mean squared error (RMSE), Nash–Sutcliffe efficiency (NSE), Kling–Gupta efficiency (KGE), and mean absolute error (MAE). The experiments over 273 watersheds show that the naïve method attains good forecasting performance (NSE > 0.5) in 88%, 65%, and 52% of watersheds at lead times of 1 day, 2 days, and 3 days, respectively. Through benchmarking by the naïve method, we find that the LSTM models trained with squared-error-based objective functions, i.e., MSE, RMSE, NSE, and KGE, perform poorly in low flow forecasting. This is because they are more influenced by training samples with high flows than by those with low flows during the model training process. For comprehensive short-term streamflow modeling without special demand orientation, we recommend the application of MAE instead of a squared-error-based metric as the objective function. In addition, it is also feasible to perform logarithmic transformation on the streamflow data. This work underscores the critical importance of appropriately selecting the objective functions for model training/calibration, shedding light on how to effectively evaluate the performance of streamflow forecast models. |
| first_indexed | 2024-04-25T00:18:44Z |
| format | Article |
| id | doaj.art-4b5b906d926b45f9bd0f59d079f4988b |
| institution | Directory Open Access Journal |
| issn | 2073-4441 |
| language | English |
| last_indexed | 2024-04-25T00:18:44Z |
| publishDate | 2024-03-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Water |
| spelling | doaj.art-4b5b906d926b45f9bd0f59d079f4988b2024-03-12T16:58:02ZengMDPI AGWater2073-44412024-03-0116577710.3390/w16050777Assessing Objective Functions in Streamflow Prediction Model Training Based on the Naïve MethodYongen Lin0Dagang Wang1Tao Jiang2Aiqing Kang3School of Geography and Planning, Sun Yat-sen University, Guangzhou 510000, ChinaSchool of Geography and Planning, Sun Yat-sen University, Guangzhou 510000, ChinaSchool of Geography and Planning, Sun Yat-sen University, Guangzhou 510000, ChinaChina Institute of Water Resources and Hydropower Research, Beijing 100038, ChinaReliable streamflow forecasting is a determining factor for water resource planning and flood control. To better understand the strengths and weaknesses of newly proposed methods in streamflow forecasting and facilitate comparisons of different research results, we test a simple, universal, and efficient benchmark method, namely, the naïve method, for short-term streamflow prediction. Using the naïve method, we assess the streamflow forecasting performance of the long short-term memory models trained with different objective functions, including mean squared error (MSE), root mean squared error (RMSE), Nash–Sutcliffe efficiency (NSE), Kling–Gupta efficiency (KGE), and mean absolute error (MAE). The experiments over 273 watersheds show that the naïve method attains good forecasting performance (NSE > 0.5) in 88%, 65%, and 52% of watersheds at lead times of 1 day, 2 days, and 3 days, respectively. Through benchmarking by the naïve method, we find that the LSTM models trained with squared-error-based objective functions, i.e., MSE, RMSE, NSE, and KGE, perform poorly in low flow forecasting. This is because they are more influenced by training samples with high flows than by those with low flows during the model training process. For comprehensive short-term streamflow modeling without special demand orientation, we recommend the application of MAE instead of a squared-error-based metric as the objective function. In addition, it is also feasible to perform logarithmic transformation on the streamflow data. This work underscores the critical importance of appropriately selecting the objective functions for model training/calibration, shedding light on how to effectively evaluate the performance of streamflow forecast models.https://www.mdpi.com/2073-4441/16/5/777streamflow predictionobjective functionmachine learningdeep learning |
| spellingShingle | Yongen Lin Dagang Wang Tao Jiang Aiqing Kang Assessing Objective Functions in Streamflow Prediction Model Training Based on the Naïve Method Water streamflow prediction objective function machine learning deep learning |
| title | Assessing Objective Functions in Streamflow Prediction Model Training Based on the Naïve Method |
| title_full | Assessing Objective Functions in Streamflow Prediction Model Training Based on the Naïve Method |
| title_fullStr | Assessing Objective Functions in Streamflow Prediction Model Training Based on the Naïve Method |
| title_full_unstemmed | Assessing Objective Functions in Streamflow Prediction Model Training Based on the Naïve Method |
| title_short | Assessing Objective Functions in Streamflow Prediction Model Training Based on the Naïve Method |
| title_sort | assessing objective functions in streamflow prediction model training based on the naive method |
| topic | streamflow prediction objective function machine learning deep learning |
| url | https://www.mdpi.com/2073-4441/16/5/777 |
| work_keys_str_mv | AT yongenlin assessingobjectivefunctionsinstreamflowpredictionmodeltrainingbasedonthenaivemethod AT dagangwang assessingobjectivefunctionsinstreamflowpredictionmodeltrainingbasedonthenaivemethod AT taojiang assessingobjectivefunctionsinstreamflowpredictionmodeltrainingbasedonthenaivemethod AT aiqingkang assessingobjectivefunctionsinstreamflowpredictionmodeltrainingbasedonthenaivemethod |