All models are wrong, but are they useful? Assessing reliability across multiple sites to build trust in urban drainage modelling
<p>Simulation models are widely used in urban drainage engineering and research, but they are known to include errors and uncertainties that are not yet fully realised. Within the herein developed framework, we investigate model adequacy across multiple sites by comparing model results with me...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Copernicus Publications
2022-11-01
|
| Series: | Hydrology and Earth System Sciences |
| Online Access: | https://hess.copernicus.org/articles/26/5879/2022/hess-26-5879-2022.pdf |
| _version_ | 1828093384170930176 |
|---|---|
| author | A. N. Pedersen A. N. Pedersen A. Brink-Kjær P. S. Mikkelsen |
| author_facet | A. N. Pedersen A. N. Pedersen A. Brink-Kjær P. S. Mikkelsen |
| author_sort | A. N. Pedersen |
| collection | DOAJ |
| description | <p>Simulation models are widely used in urban drainage engineering and research, but they are known to include errors and uncertainties that are not yet fully realised. Within the herein developed framework, we investigate model adequacy across multiple sites by comparing model results with measurements for three model objectives, namely surcharges (water level rises above defined critical levels related to basement flooding), overflows (water levels rise above a crest level), and everyday events (water levels stay below the top of pipes). We use
multi-event hydrological signatures, i.e. metrics that extract specific
characteristics of time series events in order to compare model results with the observations for the mentioned objectives through categorical and
statistical data analyses. Furthermore, we assess the events with respect to sufficient or insufficient categorical performance and good, acceptable, or poor statistical performance. We also develop a method to reduce the
weighting of individual events in the analyses, in order to acknowledge
uncertainty in model and/or measurements in cases where the model is not
expected to fully replicate the measurements. A case study including several years of water level measurements from 23 sites in two different areas shows that only few sites score a sufficient categorical performance in relation to the objective overflow and that sites do not necessarily
obtain good performance scores for all the analysed objectives. The
developed framework, however, highlights that it is possible to identify
objectives and sites for which the model is reliable, and we also suggest
methods for assessing where the model is less reliable and needs further
improvement, which may be further refined in the future.</p> |
| first_indexed | 2024-04-11T06:42:18Z |
| format | Article |
| id | doaj.art-3d19938a731b4d83861edf1c98400c5a |
| institution | Directory Open Access Journal |
| issn | 1027-5606 1607-7938 |
| language | English |
| last_indexed | 2024-04-11T06:42:18Z |
| publishDate | 2022-11-01 |
| publisher | Copernicus Publications |
| record_format | Article |
| series | Hydrology and Earth System Sciences |
| spelling | doaj.art-3d19938a731b4d83861edf1c98400c5a2022-12-22T04:39:27ZengCopernicus PublicationsHydrology and Earth System Sciences1027-56061607-79382022-11-01265879589810.5194/hess-26-5879-2022All models are wrong, but are they useful? Assessing reliability across multiple sites to build trust in urban drainage modellingA. N. Pedersen0A. N. Pedersen1A. Brink-Kjær2P. S. Mikkelsen3VCS Denmark, Vandværksvej 7, 5000 Odense C, DenmarkDTU Sustain, Technical University of Denmark, Bygningstorvet, Bygning 115, 2800 Kgs. Lyngby, DenmarkVCS Denmark, Vandværksvej 7, 5000 Odense C, DenmarkDTU Sustain, Technical University of Denmark, Bygningstorvet, Bygning 115, 2800 Kgs. Lyngby, Denmark<p>Simulation models are widely used in urban drainage engineering and research, but they are known to include errors and uncertainties that are not yet fully realised. Within the herein developed framework, we investigate model adequacy across multiple sites by comparing model results with measurements for three model objectives, namely surcharges (water level rises above defined critical levels related to basement flooding), overflows (water levels rise above a crest level), and everyday events (water levels stay below the top of pipes). We use multi-event hydrological signatures, i.e. metrics that extract specific characteristics of time series events in order to compare model results with the observations for the mentioned objectives through categorical and statistical data analyses. Furthermore, we assess the events with respect to sufficient or insufficient categorical performance and good, acceptable, or poor statistical performance. We also develop a method to reduce the weighting of individual events in the analyses, in order to acknowledge uncertainty in model and/or measurements in cases where the model is not expected to fully replicate the measurements. A case study including several years of water level measurements from 23 sites in two different areas shows that only few sites score a sufficient categorical performance in relation to the objective overflow and that sites do not necessarily obtain good performance scores for all the analysed objectives. The developed framework, however, highlights that it is possible to identify objectives and sites for which the model is reliable, and we also suggest methods for assessing where the model is less reliable and needs further improvement, which may be further refined in the future.</p>https://hess.copernicus.org/articles/26/5879/2022/hess-26-5879-2022.pdf |
| spellingShingle | A. N. Pedersen A. N. Pedersen A. Brink-Kjær P. S. Mikkelsen All models are wrong, but are they useful? Assessing reliability across multiple sites to build trust in urban drainage modelling Hydrology and Earth System Sciences |
| title | All models are wrong, but are they useful? Assessing reliability across multiple sites to build trust in urban drainage modelling |
| title_full | All models are wrong, but are they useful? Assessing reliability across multiple sites to build trust in urban drainage modelling |
| title_fullStr | All models are wrong, but are they useful? Assessing reliability across multiple sites to build trust in urban drainage modelling |
| title_full_unstemmed | All models are wrong, but are they useful? Assessing reliability across multiple sites to build trust in urban drainage modelling |
| title_short | All models are wrong, but are they useful? Assessing reliability across multiple sites to build trust in urban drainage modelling |
| title_sort | all models are wrong but are they useful assessing reliability across multiple sites to build trust in urban drainage modelling |
| url | https://hess.copernicus.org/articles/26/5879/2022/hess-26-5879-2022.pdf |
| work_keys_str_mv | AT anpedersen allmodelsarewrongbutaretheyusefulassessingreliabilityacrossmultiplesitestobuildtrustinurbandrainagemodelling AT anpedersen allmodelsarewrongbutaretheyusefulassessingreliabilityacrossmultiplesitestobuildtrustinurbandrainagemodelling AT abrinkkjær allmodelsarewrongbutaretheyusefulassessingreliabilityacrossmultiplesitestobuildtrustinurbandrainagemodelling AT psmikkelsen allmodelsarewrongbutaretheyusefulassessingreliabilityacrossmultiplesitestobuildtrustinurbandrainagemodelling |