Failure analysis of parameter-induced simulation crashes in climate models
Simulations using IPCC (Intergovernmental Panel on Climate Change)-class climate models are subject to fail or crash for a variety of reasons. Quantitative analysis of the failures can yield useful insights to better understand and improve the models. During the course of uncertainty quantificatio...
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Copernicus Publications
2013-08-01
|
| Series: | Geoscientific Model Development |
| Online Access: | http://www.geosci-model-dev.net/6/1157/2013/gmd-6-1157-2013.pdf |
| _version_ | 1818979846667108352 |
|---|---|
| author | D. D. Lucas R. Klein J. Tannahill D. Ivanova S. Brandon D. Domyancic Y. Zhang |
| author_facet | D. D. Lucas R. Klein J. Tannahill D. Ivanova S. Brandon D. Domyancic Y. Zhang |
| author_sort | D. D. Lucas |
| collection | DOAJ |
| description | Simulations using IPCC (Intergovernmental Panel on Climate Change)-class climate models are subject to fail or crash for a variety of reasons. Quantitative analysis of the failures can yield useful insights to better understand and improve the models. During the course of uncertainty quantification (UQ) ensemble simulations to assess the effects of ocean model parameter uncertainties on climate simulations, we experienced a series of simulation crashes within the Parallel Ocean Program (POP2) component of the Community Climate System Model (CCSM4). About 8.5% of our CCSM4 simulations failed for numerical reasons at combinations of POP2 parameter values. We applied support vector machine (SVM) classification from machine learning to quantify and predict the probability of failure as a function of the values of 18 POP2 parameters. A committee of SVM classifiers readily predicted model failures in an independent validation ensemble, as assessed by the area under the receiver operating characteristic (ROC) curve metric (AUC > 0.96). The causes of the simulation failures were determined through a global sensitivity analysis. Combinations of 8 parameters related to ocean mixing and viscosity from three different POP2 parameterizations were the major sources of the failures. This information can be used to improve POP2 and CCSM4 by incorporating correlations across the relevant parameters. Our method can also be used to quantify, predict, and understand simulation crashes in other complex geoscientific models. |
| first_indexed | 2024-12-20T17:06:01Z |
| format | Article |
| id | doaj.art-c5d3e8cb0efe43f9b98f6ec3ccafe940 |
| institution | Directory Open Access Journal |
| issn | 1991-959X 1991-9603 |
| language | English |
| last_indexed | 2024-12-20T17:06:01Z |
| publishDate | 2013-08-01 |
| publisher | Copernicus Publications |
| record_format | Article |
| series | Geoscientific Model Development |
| spelling | doaj.art-c5d3e8cb0efe43f9b98f6ec3ccafe9402022-12-21T19:32:18ZengCopernicus PublicationsGeoscientific Model Development1991-959X1991-96032013-08-01641157117110.5194/gmd-6-1157-2013Failure analysis of parameter-induced simulation crashes in climate modelsD. D. LucasR. KleinJ. TannahillD. IvanovaS. BrandonD. DomyancicY. ZhangSimulations using IPCC (Intergovernmental Panel on Climate Change)-class climate models are subject to fail or crash for a variety of reasons. Quantitative analysis of the failures can yield useful insights to better understand and improve the models. During the course of uncertainty quantification (UQ) ensemble simulations to assess the effects of ocean model parameter uncertainties on climate simulations, we experienced a series of simulation crashes within the Parallel Ocean Program (POP2) component of the Community Climate System Model (CCSM4). About 8.5% of our CCSM4 simulations failed for numerical reasons at combinations of POP2 parameter values. We applied support vector machine (SVM) classification from machine learning to quantify and predict the probability of failure as a function of the values of 18 POP2 parameters. A committee of SVM classifiers readily predicted model failures in an independent validation ensemble, as assessed by the area under the receiver operating characteristic (ROC) curve metric (AUC > 0.96). The causes of the simulation failures were determined through a global sensitivity analysis. Combinations of 8 parameters related to ocean mixing and viscosity from three different POP2 parameterizations were the major sources of the failures. This information can be used to improve POP2 and CCSM4 by incorporating correlations across the relevant parameters. Our method can also be used to quantify, predict, and understand simulation crashes in other complex geoscientific models.http://www.geosci-model-dev.net/6/1157/2013/gmd-6-1157-2013.pdf |
| spellingShingle | D. D. Lucas R. Klein J. Tannahill D. Ivanova S. Brandon D. Domyancic Y. Zhang Failure analysis of parameter-induced simulation crashes in climate models Geoscientific Model Development |
| title | Failure analysis of parameter-induced simulation crashes in climate models |
| title_full | Failure analysis of parameter-induced simulation crashes in climate models |
| title_fullStr | Failure analysis of parameter-induced simulation crashes in climate models |
| title_full_unstemmed | Failure analysis of parameter-induced simulation crashes in climate models |
| title_short | Failure analysis of parameter-induced simulation crashes in climate models |
| title_sort | failure analysis of parameter induced simulation crashes in climate models |
| url | http://www.geosci-model-dev.net/6/1157/2013/gmd-6-1157-2013.pdf |
| work_keys_str_mv | AT ddlucas failureanalysisofparameterinducedsimulationcrashesinclimatemodels AT rklein failureanalysisofparameterinducedsimulationcrashesinclimatemodels AT jtannahill failureanalysisofparameterinducedsimulationcrashesinclimatemodels AT divanova failureanalysisofparameterinducedsimulationcrashesinclimatemodels AT sbrandon failureanalysisofparameterinducedsimulationcrashesinclimatemodels AT ddomyancic failureanalysisofparameterinducedsimulationcrashesinclimatemodels AT yzhang failureanalysisofparameterinducedsimulationcrashesinclimatemodels |