Applying the FAHP to Improve the Performance Evaluation Reliability of Software Defect Classifiers
Today's software complexity makes developing defect-free software almost impossible. Consequently, developing classifiers to classify software modules into defective and non-defective before software releases have attracted great interest in academia and software industry alike. Although many c...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2019-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/8710236/ |
| _version_ | 1819179171588341760 |
|---|---|
| author | Hussam Ghunaim Julius Dichter |
| author_facet | Hussam Ghunaim Julius Dichter |
| author_sort | Hussam Ghunaim |
| collection | DOAJ |
| description | Today's software complexity makes developing defect-free software almost impossible. Consequently, developing classifiers to classify software modules into defective and non-defective before software releases have attracted great interest in academia and software industry alike. Although many classifiers have been proposed, no one has been proven superior over others. The major reason is that while a research shows that classifier A is better than classifier B, we can find other research that shows the opposite. These conflicts are usually triggered when researchers report results using their preferable performance evaluation measures such as, recall and precision. Although this approach is valid, it does not examine all possible facets of classifiers performance characteristics. Thus, the performance evaluation might improve or deteriorate if researchers choose other performance measures. As a result, software developers usually struggle to select the most suitable classifier to use in their projects. The goal of this paper is to apply the fuzzy analytical hierarchy process (FAHP) as a popular multicriteria decision-making technique to reliably evaluate classifiers' performance. This evaluation framework incorporates a wider spectrum of performance measures to evaluate classifiers performance rather than relying on selected preferable measures. The results show that this approach will increase software developers' confidence in research outcomes and help them in avoiding false conclusions and infer reasonable boundaries for them. We exploited 22 popular performance measures and 11 software defect classifiers. The analysis was carried out using KNIME data mining platform and 12 software defect data sets provided by the NASA metrics data program (MDP) repository. |
| first_indexed | 2024-12-22T21:54:12Z |
| format | Article |
| id | doaj.art-373e29e9472b4a4193e8e40917e6286e |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-12-22T21:54:12Z |
| publishDate | 2019-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-373e29e9472b4a4193e8e40917e6286e2022-12-21T18:11:16ZengIEEEIEEE Access2169-35362019-01-017627946280410.1109/ACCESS.2019.29159648710236Applying the FAHP to Improve the Performance Evaluation Reliability of Software Defect ClassifiersHussam Ghunaim0https://orcid.org/0000-0002-2859-2189Julius Dichter1Computer Science and Engineering Department, School of Engineering, University of Bridgeport, Bridgeport, CT, USAComputer Science and Engineering Department, School of Engineering, University of Bridgeport, Bridgeport, CT, USAToday's software complexity makes developing defect-free software almost impossible. Consequently, developing classifiers to classify software modules into defective and non-defective before software releases have attracted great interest in academia and software industry alike. Although many classifiers have been proposed, no one has been proven superior over others. The major reason is that while a research shows that classifier A is better than classifier B, we can find other research that shows the opposite. These conflicts are usually triggered when researchers report results using their preferable performance evaluation measures such as, recall and precision. Although this approach is valid, it does not examine all possible facets of classifiers performance characteristics. Thus, the performance evaluation might improve or deteriorate if researchers choose other performance measures. As a result, software developers usually struggle to select the most suitable classifier to use in their projects. The goal of this paper is to apply the fuzzy analytical hierarchy process (FAHP) as a popular multicriteria decision-making technique to reliably evaluate classifiers' performance. This evaluation framework incorporates a wider spectrum of performance measures to evaluate classifiers performance rather than relying on selected preferable measures. The results show that this approach will increase software developers' confidence in research outcomes and help them in avoiding false conclusions and infer reasonable boundaries for them. We exploited 22 popular performance measures and 11 software defect classifiers. The analysis was carried out using KNIME data mining platform and 12 software defect data sets provided by the NASA metrics data program (MDP) repository.https://ieeexplore.ieee.org/document/8710236/Classifiersdata miningempirical software engineering (ESE)FAHPKNIMEperformance evaluation |
| spellingShingle | Hussam Ghunaim Julius Dichter Applying the FAHP to Improve the Performance Evaluation Reliability of Software Defect Classifiers IEEE Access Classifiers data mining empirical software engineering (ESE) FAHP KNIME performance evaluation |
| title | Applying the FAHP to Improve the Performance Evaluation Reliability of Software Defect Classifiers |
| title_full | Applying the FAHP to Improve the Performance Evaluation Reliability of Software Defect Classifiers |
| title_fullStr | Applying the FAHP to Improve the Performance Evaluation Reliability of Software Defect Classifiers |
| title_full_unstemmed | Applying the FAHP to Improve the Performance Evaluation Reliability of Software Defect Classifiers |
| title_short | Applying the FAHP to Improve the Performance Evaluation Reliability of Software Defect Classifiers |
| title_sort | applying the fahp to improve the performance evaluation reliability of software defect classifiers |
| topic | Classifiers data mining empirical software engineering (ESE) FAHP KNIME performance evaluation |
| url | https://ieeexplore.ieee.org/document/8710236/ |
| work_keys_str_mv | AT hussamghunaim applyingthefahptoimprovetheperformanceevaluationreliabilityofsoftwaredefectclassifiers AT juliusdichter applyingthefahptoimprovetheperformanceevaluationreliabilityofsoftwaredefectclassifiers |