Metrics for Offline Evaluation of Prognostic Performance
Prognostic performance evaluation has gained significant attention in the past few years.Currently, prognostics concepts lack standard definitions and suffer from ambiguous and inconsistent interpretations. This lack of standards is in part due to the varied end-user requirements for different appli...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
The Prognostics and Health Management Society
2010-01-01
|
| Series: | International Journal of Prognostics and Health Management |
| Subjects: | |
| Online Access: | http://www.phmsociety.org/sites/all/modules/pubdlcnt/pubdlcnt.php?file=http://www.phmsociety.org/sites/phmsociety.org/files/phm_submission/2010/ijPHM_10_001.pdf&nid=263 |
| _version_ | 1818613777041457152 |
|---|---|
| author | Sankalita Saha Bhaskar Saha Abhinav Saxena Kai Goebel Jose Celaya |
| author_facet | Sankalita Saha Bhaskar Saha Abhinav Saxena Kai Goebel Jose Celaya |
| author_sort | Sankalita Saha |
| collection | DOAJ |
| description | Prognostic performance evaluation has gained significant attention in the past few years.Currently, prognostics concepts lack standard definitions and suffer from ambiguous and inconsistent interpretations. This lack of standards is in part due to the varied end-user requirements for different applications, time scales, available information, domain dynamics, etc. to name a few. The research community has used a variety of metrics largely based on convenience and their respective requirements. Very little attention has been focused on establishing a standardized approach to compare different efforts. This paper presents several new evaluation metrics tailored for prognostics that were recently introduced and were shown to effectively evaluate various algorithms as compared to other conventional metrics. Specifically, this paper presents a detailed discussion on how these metrics should be interpreted and used. These metrics have the capability of incorporating probabilistic uncertainty estimates from prognostic algorithms. In addition to quantitative assessment they also offer a comprehensive visual perspective that can be used in designing the prognostic system. Several methods are suggested to customize these metrics for different applications. Guidelines are provided to help choose one method over another based on distribution characteristics. Various issues faced by prognostics and its performance evaluation are discussed followed by a formal notational framework to help standardize subsequent developments. |
| first_indexed | 2024-12-16T16:07:30Z |
| format | Article |
| id | doaj.art-566d2058aef140e780e873ac3b224ffd |
| institution | Directory Open Access Journal |
| issn | 2153-2648 |
| language | English |
| last_indexed | 2024-12-16T16:07:30Z |
| publishDate | 2010-01-01 |
| publisher | The Prognostics and Health Management Society |
| record_format | Article |
| series | International Journal of Prognostics and Health Management |
| spelling | doaj.art-566d2058aef140e780e873ac3b224ffd2022-12-21T22:25:19ZengThe Prognostics and Health Management SocietyInternational Journal of Prognostics and Health Management2153-26482010-01-0111120Metrics for Offline Evaluation of Prognostic PerformanceSankalita SahaBhaskar SahaAbhinav SaxenaKai GoebelJose CelayaPrognostic performance evaluation has gained significant attention in the past few years.Currently, prognostics concepts lack standard definitions and suffer from ambiguous and inconsistent interpretations. This lack of standards is in part due to the varied end-user requirements for different applications, time scales, available information, domain dynamics, etc. to name a few. The research community has used a variety of metrics largely based on convenience and their respective requirements. Very little attention has been focused on establishing a standardized approach to compare different efforts. This paper presents several new evaluation metrics tailored for prognostics that were recently introduced and were shown to effectively evaluate various algorithms as compared to other conventional metrics. Specifically, this paper presents a detailed discussion on how these metrics should be interpreted and used. These metrics have the capability of incorporating probabilistic uncertainty estimates from prognostic algorithms. In addition to quantitative assessment they also offer a comprehensive visual perspective that can be used in designing the prognostic system. Several methods are suggested to customize these metrics for different applications. Guidelines are provided to help choose one method over another based on distribution characteristics. Various issues faced by prognostics and its performance evaluation are discussed followed by a formal notational framework to help standardize subsequent developments.http://www.phmsociety.org/sites/all/modules/pubdlcnt/pubdlcnt.php?file=http://www.phmsociety.org/sites/phmsociety.org/files/phm_submission/2010/ijPHM_10_001.pdf&nid=263preventive maintenanceprognostic performanceprognosticsremaining useful life (RUL) |
| spellingShingle | Sankalita Saha Bhaskar Saha Abhinav Saxena Kai Goebel Jose Celaya Metrics for Offline Evaluation of Prognostic Performance International Journal of Prognostics and Health Management preventive maintenance prognostic performance prognostics remaining useful life (RUL) |
| title | Metrics for Offline Evaluation of Prognostic Performance |
| title_full | Metrics for Offline Evaluation of Prognostic Performance |
| title_fullStr | Metrics for Offline Evaluation of Prognostic Performance |
| title_full_unstemmed | Metrics for Offline Evaluation of Prognostic Performance |
| title_short | Metrics for Offline Evaluation of Prognostic Performance |
| title_sort | metrics for offline evaluation of prognostic performance |
| topic | preventive maintenance prognostic performance prognostics remaining useful life (RUL) |
| url | http://www.phmsociety.org/sites/all/modules/pubdlcnt/pubdlcnt.php?file=http://www.phmsociety.org/sites/phmsociety.org/files/phm_submission/2010/ijPHM_10_001.pdf&nid=263 |
| work_keys_str_mv | AT sankalitasaha metricsforofflineevaluationofprognosticperformance AT bhaskarsaha metricsforofflineevaluationofprognosticperformance AT abhinavsaxena metricsforofflineevaluationofprognosticperformance AT kaigoebel metricsforofflineevaluationofprognosticperformance AT josecelaya metricsforofflineevaluationofprognosticperformance |