A general model validation and testing tool
We construct and propose the “Bayesian Validation Metric” (BVM) as a general model validation and testing tool. We find the BVM to be capable of representing all of the standard validation metrics (square error, reliability, probability of agreement, frequentist, area, probability density comparison...
| Main Authors: | , , |
|---|---|
| Other Authors: | |
| Format: | Article |
| Language: | English |
| Published: |
Elsevier BV
2020
|
| Online Access: | https://hdl.handle.net/1721.1/127676 |
| _version_ | 1826209732991385600 |
|---|---|
| author | Vanslette, Kevin Tohme, Tony. Youcef-Toumi, Kamal |
| author2 | Massachusetts Institute of Technology. Department of Mechanical Engineering |
| author_facet | Massachusetts Institute of Technology. Department of Mechanical Engineering Vanslette, Kevin Tohme, Tony. Youcef-Toumi, Kamal |
| author_sort | Vanslette, Kevin |
| collection | MIT |
| description | We construct and propose the “Bayesian Validation Metric” (BVM) as a general model validation and testing tool. We find the BVM to be capable of representing all of the standard validation metrics (square error, reliability, probability of agreement, frequentist, area, probability density comparison, statistical hypothesis testing, and Bayesian model testing) as special cases and find that it can be used to improve, generalize, or further quantify their uncertainties. Thus, the BVM allows us to assess the similarities and differences between existing validation metrics in a new light. The BVM has the capacity to allow users to invent and select models according to novel validation requirements. We formulate and test a few novel compound validation metrics that improve upon other validation metrics in the literature. Further, we construct the BVM Ratio for the purpose of quantifying model selection under user defined definitions of agreement in the presence or absence of uncertainty. This construction generalizes the Bayesian model testing framework. ©2019 Elsevier Ltd |
| first_indexed | 2024-09-23T14:28:09Z |
| format | Article |
| id | mit-1721.1/127676 |
| institution | Massachusetts Institute of Technology |
| language | English |
| last_indexed | 2024-09-23T14:28:09Z |
| publishDate | 2020 |
| publisher | Elsevier BV |
| record_format | dspace |
| spelling | mit-1721.1/1276762022-10-01T21:27:37Z A general model validation and testing tool Vanslette, Kevin Tohme, Tony. Youcef-Toumi, Kamal Massachusetts Institute of Technology. Department of Mechanical Engineering We construct and propose the “Bayesian Validation Metric” (BVM) as a general model validation and testing tool. We find the BVM to be capable of representing all of the standard validation metrics (square error, reliability, probability of agreement, frequentist, area, probability density comparison, statistical hypothesis testing, and Bayesian model testing) as special cases and find that it can be used to improve, generalize, or further quantify their uncertainties. Thus, the BVM allows us to assess the similarities and differences between existing validation metrics in a new light. The BVM has the capacity to allow users to invent and select models according to novel validation requirements. We formulate and test a few novel compound validation metrics that improve upon other validation metrics in the literature. Further, we construct the BVM Ratio for the purpose of quantifying model selection under user defined definitions of agreement in the presence or absence of uncertainty. This construction generalizes the Bayesian model testing framework. ©2019 Elsevier Ltd 2020-09-22T16:38:36Z 2020-09-22T16:38:36Z 2019-10 2019-09 2020-08-14T14:23:07Z Article http://purl.org/eprint/type/JournalArticle 1879-0836 https://hdl.handle.net/1721.1/127676 Vanslette, Kevin et al., "A general model validation and testing tool." Reliability Engineering & System Safety 195 (March 2020): 106684 doi. 10.1016/j.ress.2019.106684 ©2019 en https://dx.doi.org/10.1016/J.RESS.2019.106684 Reliability Engineering & System Safety Creative Commons Attribution-NonCommercial-NoDerivs License http://creativecommons.org/licenses/by-nc-nd/4.0/ application/pdf Elsevier BV arXiv |
| spellingShingle | Vanslette, Kevin Tohme, Tony. Youcef-Toumi, Kamal A general model validation and testing tool |
| title | A general model validation and testing tool |
| title_full | A general model validation and testing tool |
| title_fullStr | A general model validation and testing tool |
| title_full_unstemmed | A general model validation and testing tool |
| title_short | A general model validation and testing tool |
| title_sort | general model validation and testing tool |
| url | https://hdl.handle.net/1721.1/127676 |
| work_keys_str_mv | AT vanslettekevin ageneralmodelvalidationandtestingtool AT tohmetony ageneralmodelvalidationandtestingtool AT youceftoumikamal ageneralmodelvalidationandtestingtool AT vanslettekevin generalmodelvalidationandtestingtool AT tohmetony generalmodelvalidationandtestingtool AT youceftoumikamal generalmodelvalidationandtestingtool |