A Hadoop cloud-based surrogate modelling framework for approximating complex hydrological models
Hydrological simulation has long been a challenge because of the computationally intensive and expensive nature of complex hydrological models. In this paper, a surrogate modelling (SM) framework is presented based on the Hadoop cloud for approximating complex hydrological models. The substantial mo...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IWA Publishing
2023-03-01
|
| Series: | Journal of Hydroinformatics |
| Subjects: | |
| Online Access: | http://jhydro.iwaponline.com/content/25/2/511 |
| _version_ | 1797200764921184256 |
|---|---|
| author | Jinfeng Ma Hua Zheng Ruonan Li Kaifeng Rao Yanzheng Yang Weifeng Li |
| author_facet | Jinfeng Ma Hua Zheng Ruonan Li Kaifeng Rao Yanzheng Yang Weifeng Li |
| author_sort | Jinfeng Ma |
| collection | DOAJ |
| description | Hydrological simulation has long been a challenge because of the computationally intensive and expensive nature of complex hydrological models. In this paper, a surrogate modelling (SM) framework is presented based on the Hadoop cloud for approximating complex hydrological models. The substantial model runs required by the design of the experiment (DOE) of SM were solved using the Hadoop cloud. Polynomial chaos expansion (PCE) was fitted and verified using the high-fidelity model DOE and was then used as a case study to investigate the approximation capability in a Soil and Water Assessment Tool (SWAT) surrogate model with regard to the accuracy, fidelity, and efficiency. In experiments, the Hadoop cloud reduced the computation time by approximately 86% when used in a global sensitivity analysis. PCE achieved results equivalent to those of the standard Monte Carlo approach, with a flow variance coefficient of determination of 0.92. Moreover, PCE proved to be as reliable as the Monte Carlo approach but significantly more efficient. The proposed framework greatly decreases the computational costs through cloud computing and surrogate modelling, making it ideal for complex hydrological model simulation and optimization.
HIGHLIGHTS
Our surrogate modelling framework reduces the computational cost of simulations.;
The design of the experiment was parallelized on a Hadoop cloud.;
PCE was fitted and verified using a high-fidelity model.;
The approximation ability of PCE in the SWAT surrogate model was investigated.; |
| first_indexed | 2024-04-09T19:05:18Z |
| format | Article |
| id | doaj.art-ebac47a3f057459ba568f70c32d30eee |
| institution | Directory Open Access Journal |
| issn | 1464-7141 1465-1734 |
| language | English |
| last_indexed | 2024-04-24T07:36:50Z |
| publishDate | 2023-03-01 |
| publisher | IWA Publishing |
| record_format | Article |
| series | Journal of Hydroinformatics |
| spelling | doaj.art-ebac47a3f057459ba568f70c32d30eee2024-04-20T06:20:30ZengIWA PublishingJournal of Hydroinformatics1464-71411465-17342023-03-0125251152510.2166/hydro.2023.184184A Hadoop cloud-based surrogate modelling framework for approximating complex hydrological modelsJinfeng Ma0Hua Zheng1Ruonan Li2Kaifeng Rao3Yanzheng Yang4Weifeng Li5 Research Centre for Eco-Environmental Sciences Chinese Academy of Sciences, Beijing 100085, China Research Centre for Eco-Environmental Sciences Chinese Academy of Sciences, Beijing 100085, China Research Centre for Eco-Environmental Sciences Chinese Academy of Sciences, Beijing 100085, China Research Centre for Eco-Environmental Sciences Chinese Academy of Sciences, Beijing 100085, China Research Centre for Eco-Environmental Sciences Chinese Academy of Sciences, Beijing 100085, China Research Centre for Eco-Environmental Sciences Chinese Academy of Sciences, Beijing 100085, China Hydrological simulation has long been a challenge because of the computationally intensive and expensive nature of complex hydrological models. In this paper, a surrogate modelling (SM) framework is presented based on the Hadoop cloud for approximating complex hydrological models. The substantial model runs required by the design of the experiment (DOE) of SM were solved using the Hadoop cloud. Polynomial chaos expansion (PCE) was fitted and verified using the high-fidelity model DOE and was then used as a case study to investigate the approximation capability in a Soil and Water Assessment Tool (SWAT) surrogate model with regard to the accuracy, fidelity, and efficiency. In experiments, the Hadoop cloud reduced the computation time by approximately 86% when used in a global sensitivity analysis. PCE achieved results equivalent to those of the standard Monte Carlo approach, with a flow variance coefficient of determination of 0.92. Moreover, PCE proved to be as reliable as the Monte Carlo approach but significantly more efficient. The proposed framework greatly decreases the computational costs through cloud computing and surrogate modelling, making it ideal for complex hydrological model simulation and optimization. HIGHLIGHTS Our surrogate modelling framework reduces the computational cost of simulations.; The design of the experiment was parallelized on a Hadoop cloud.; PCE was fitted and verified using a high-fidelity model.; The approximation ability of PCE in the SWAT surrogate model was investigated.;http://jhydro.iwaponline.com/content/25/2/511chaospyhadoop cloudpolynomial chaos expansionsurrogate modellingswat |
| spellingShingle | Jinfeng Ma Hua Zheng Ruonan Li Kaifeng Rao Yanzheng Yang Weifeng Li A Hadoop cloud-based surrogate modelling framework for approximating complex hydrological models Journal of Hydroinformatics chaospy hadoop cloud polynomial chaos expansion surrogate modelling swat |
| title | A Hadoop cloud-based surrogate modelling framework for approximating complex hydrological models |
| title_full | A Hadoop cloud-based surrogate modelling framework for approximating complex hydrological models |
| title_fullStr | A Hadoop cloud-based surrogate modelling framework for approximating complex hydrological models |
| title_full_unstemmed | A Hadoop cloud-based surrogate modelling framework for approximating complex hydrological models |
| title_short | A Hadoop cloud-based surrogate modelling framework for approximating complex hydrological models |
| title_sort | hadoop cloud based surrogate modelling framework for approximating complex hydrological models |
| topic | chaospy hadoop cloud polynomial chaos expansion surrogate modelling swat |
| url | http://jhydro.iwaponline.com/content/25/2/511 |
| work_keys_str_mv | AT jinfengma ahadoopcloudbasedsurrogatemodellingframeworkforapproximatingcomplexhydrologicalmodels AT huazheng ahadoopcloudbasedsurrogatemodellingframeworkforapproximatingcomplexhydrologicalmodels AT ruonanli ahadoopcloudbasedsurrogatemodellingframeworkforapproximatingcomplexhydrologicalmodels AT kaifengrao ahadoopcloudbasedsurrogatemodellingframeworkforapproximatingcomplexhydrologicalmodels AT yanzhengyang ahadoopcloudbasedsurrogatemodellingframeworkforapproximatingcomplexhydrologicalmodels AT weifengli ahadoopcloudbasedsurrogatemodellingframeworkforapproximatingcomplexhydrologicalmodels AT jinfengma hadoopcloudbasedsurrogatemodellingframeworkforapproximatingcomplexhydrologicalmodels AT huazheng hadoopcloudbasedsurrogatemodellingframeworkforapproximatingcomplexhydrologicalmodels AT ruonanli hadoopcloudbasedsurrogatemodellingframeworkforapproximatingcomplexhydrologicalmodels AT kaifengrao hadoopcloudbasedsurrogatemodellingframeworkforapproximatingcomplexhydrologicalmodels AT yanzhengyang hadoopcloudbasedsurrogatemodellingframeworkforapproximatingcomplexhydrologicalmodels AT weifengli hadoopcloudbasedsurrogatemodellingframeworkforapproximatingcomplexhydrologicalmodels |