Key challenges to model-based design : distinguishing model confidence from model validation
Thesis (S.M. in Engineering and Management)--Massachusetts Institute of Technology, Engineering Systems Division, System Design and Management Program, 2012.
| Main Author: | |
|---|---|
| Other Authors: | |
| Format: | Thesis |
| Language: | eng |
| Published: |
Massachusetts Institute of Technology
2013
|
| Subjects: | |
| Online Access: | http://hdl.handle.net/1721.1/76492 |
| _version_ | 1811069707540758528 |
|---|---|
| author | Flanagan, Genevieve (Genevieve Elise Cregar) |
| author2 | Olivier L. de Weck and Noelle Eckley Selin. |
| author_facet | Olivier L. de Weck and Noelle Eckley Selin. Flanagan, Genevieve (Genevieve Elise Cregar) |
| author_sort | Flanagan, Genevieve (Genevieve Elise Cregar) |
| collection | MIT |
| description | Thesis (S.M. in Engineering and Management)--Massachusetts Institute of Technology, Engineering Systems Division, System Design and Management Program, 2012. |
| first_indexed | 2024-09-23T08:14:39Z |
| format | Thesis |
| id | mit-1721.1/76492 |
| institution | Massachusetts Institute of Technology |
| language | eng |
| last_indexed | 2024-09-23T08:14:39Z |
| publishDate | 2013 |
| publisher | Massachusetts Institute of Technology |
| record_format | dspace |
| spelling | mit-1721.1/764922019-04-09T17:32:00Z Key challenges to model-based design : distinguishing model confidence from model validation Flanagan, Genevieve (Genevieve Elise Cregar) Olivier L. de Weck and Noelle Eckley Selin. Massachusetts Institute of Technology. Engineering Systems Division. System Design and Management Program. Massachusetts Institute of Technology. Engineering Systems Division. System Design and Management Program. Engineering Systems Division. Thesis (S.M. in Engineering and Management)--Massachusetts Institute of Technology, Engineering Systems Division, System Design and Management Program, 2012. Cataloged from PDF version of thesis. Includes bibliographical references (p. 93-97). Model-based design is becoming more prevalent in industry due to increasing complexities in technology while schedules shorten and budgets tighten. Model-based design is a means to substantiate good design under these circumstances. Despite this, organizations often have a lack of confidence in the use of models to make critical decisions. As a consequence they often invest heavily in expensive test activities that may not yield substantially new or better information. On the other hand, models are often used beyond the bounds within which they had been previously calibrated and validated and their predictions in the new regime may be substantially in error and this can add substantial risk to a program. This thesis seeks to identify factors that cause either of these behaviors. Eight factors emerged as the key variables to misaligned model confidence. These were found by studying three case studies to setup the problem space. This was followed by a review of the literature with emphasis on model validation and assessment processes to identify remaining gaps. These gaps include proper model validation processes, limited research from the perspective of the decision-maker, and lack of understanding of the impact of contextual variables surrounding a decision. The impact these eight factors have on model confidence and credibility was tested using a web-based experiment that included a simple model of a catapult and varying contextual details representing the factors. In total 252 respondents interacted with the model and made a binary decision on a design problem to provide a measure for model confidence. Results from the testing showed several factors proved to cause an outright change in model confidence. One factor, a representation of model uncertainty, did not result in any differences to model confidence despite support from the literature suggesting otherwise. Findings such as these were used to gain additional insights and recommendations to address the problem of misaligned model confidence. Recommendations included system-level approaches, improved quality of communication, and use of decision analysis techniques. Applying focus in these areas can help to alleviate pressures from the contextual factors involved in the decision-making process. This will allow models to be used more effectively thereby supporting model-based design efforts. by Genevieve Flanagan. S.M.in Engineering and Management 2013-01-23T19:44:02Z 2013-01-23T19:44:02Z 2012 2012 Thesis http://hdl.handle.net/1721.1/76492 822586793 eng M.I.T. theses are protected by copyright. They may be viewed from this source for any purpose, but reproduction or distribution in any format is prohibited without written permission. See provided URL for inquiries about permission. http://dspace.mit.edu/handle/1721.1/7582 133 p. application/pdf Massachusetts Institute of Technology |
| spellingShingle | System Design and Management Program. Engineering Systems Division. Flanagan, Genevieve (Genevieve Elise Cregar) Key challenges to model-based design : distinguishing model confidence from model validation |
| title | Key challenges to model-based design : distinguishing model confidence from model validation |
| title_full | Key challenges to model-based design : distinguishing model confidence from model validation |
| title_fullStr | Key challenges to model-based design : distinguishing model confidence from model validation |
| title_full_unstemmed | Key challenges to model-based design : distinguishing model confidence from model validation |
| title_short | Key challenges to model-based design : distinguishing model confidence from model validation |
| title_sort | key challenges to model based design distinguishing model confidence from model validation |
| topic | System Design and Management Program. Engineering Systems Division. |
| url | http://hdl.handle.net/1721.1/76492 |
| work_keys_str_mv | AT flanagangenevievegenevieveelisecregar keychallengestomodelbaseddesigndistinguishingmodelconfidencefrommodelvalidation |