CODE: A Moving-Window-Based Framework for Detecting Concept Drift in Software Defect Prediction
Concept drift (CD) refers to data distributions that may vary after a minimum stable period. CD negatively influences models’ performance of software defect prediction (SDP) trained on past datasets when applied to the new datasets. Based on previous studies of SDP, it is confirmed that the accuracy...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2022-11-01
|
| Series: | Symmetry |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2073-8994/14/12/2508 |
| _version_ | 1797455200522338304 |
|---|---|
| author | Md Alamgir Kabir Shahina Begum Mobyen Uddin Ahmed Atiq Ur Rehman |
| author_facet | Md Alamgir Kabir Shahina Begum Mobyen Uddin Ahmed Atiq Ur Rehman |
| author_sort | Md Alamgir Kabir |
| collection | DOAJ |
| description | Concept drift (CD) refers to data distributions that may vary after a minimum stable period. CD negatively influences models’ performance of software defect prediction (SDP) trained on past datasets when applied to the new datasets. Based on previous studies of SDP, it is confirmed that the accuracy of prediction models is negatively affected due to changes in data distributions. Moreover, cross-version (CV) defect data are naturally asymmetric due to the nature of their class imbalance. In this paper, a moving window-based concept-drift detection (CODE) framework is proposed to detect CD in chronologically asymmetric defective datasets and to investigate the feasibility of alleviating CD from the data. The proposed CODE framework consists of four steps, in which the first pre-processes the defect datasets and forms CV chronological data, the second constructs the CV defect models, the third calculates the test statistics, and the fourth provides a hypothesis-test-based CD detection method. In prior studies of SDP, it is observed that in an effort to make the data more symmetric, class-rebalancing techniques are utilized, and this improves the prediction performance of the models. The ability of the CODE framework is demonstrated by conducting experiments on 36 versions of 10 software projects. Some of the key findings are: (1) Up to 50% of the chronological-defect datasets are drift-prone while applying the most popular classifiers used from the SDP literature. (2) The class-rebalancing techniques had a positive impact on the prediction performance for CVDP by correctly classifying the CV defective modules and detected CD by up to 31% on the resampled datasets. |
| first_indexed | 2024-03-09T15:49:07Z |
| format | Article |
| id | doaj.art-dbf4240119d641749d1814f5fdcfd3e2 |
| institution | Directory Open Access Journal |
| issn | 2073-8994 |
| language | English |
| last_indexed | 2024-03-09T15:49:07Z |
| publishDate | 2022-11-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Symmetry |
| spelling | doaj.art-dbf4240119d641749d1814f5fdcfd3e22023-11-24T18:18:58ZengMDPI AGSymmetry2073-89942022-11-011412250810.3390/sym14122508CODE: A Moving-Window-Based Framework for Detecting Concept Drift in Software Defect PredictionMd Alamgir Kabir0Shahina Begum1Mobyen Uddin Ahmed2Atiq Ur Rehman3Artificial Intelligence and Intelligent Systems Research Group, School of Innovation, Design and Engineering, Mälardalen University, Högskoleplan 1, 722 20 Västerås, SwedenArtificial Intelligence and Intelligent Systems Research Group, School of Innovation, Design and Engineering, Mälardalen University, Högskoleplan 1, 722 20 Västerås, SwedenArtificial Intelligence and Intelligent Systems Research Group, School of Innovation, Design and Engineering, Mälardalen University, Högskoleplan 1, 722 20 Västerås, SwedenArtificial Intelligence and Intelligent Systems Research Group, School of Innovation, Design and Engineering, Mälardalen University, Högskoleplan 1, 722 20 Västerås, SwedenConcept drift (CD) refers to data distributions that may vary after a minimum stable period. CD negatively influences models’ performance of software defect prediction (SDP) trained on past datasets when applied to the new datasets. Based on previous studies of SDP, it is confirmed that the accuracy of prediction models is negatively affected due to changes in data distributions. Moreover, cross-version (CV) defect data are naturally asymmetric due to the nature of their class imbalance. In this paper, a moving window-based concept-drift detection (CODE) framework is proposed to detect CD in chronologically asymmetric defective datasets and to investigate the feasibility of alleviating CD from the data. The proposed CODE framework consists of four steps, in which the first pre-processes the defect datasets and forms CV chronological data, the second constructs the CV defect models, the third calculates the test statistics, and the fourth provides a hypothesis-test-based CD detection method. In prior studies of SDP, it is observed that in an effort to make the data more symmetric, class-rebalancing techniques are utilized, and this improves the prediction performance of the models. The ability of the CODE framework is demonstrated by conducting experiments on 36 versions of 10 software projects. Some of the key findings are: (1) Up to 50% of the chronological-defect datasets are drift-prone while applying the most popular classifiers used from the SDP literature. (2) The class-rebalancing techniques had a positive impact on the prediction performance for CVDP by correctly classifying the CV defective modules and detected CD by up to 31% on the resampled datasets.https://www.mdpi.com/2073-8994/14/12/2508software defect predictioncross-version defect predictionchronological-defect dataclass rebalancingconcept drift |
| spellingShingle | Md Alamgir Kabir Shahina Begum Mobyen Uddin Ahmed Atiq Ur Rehman CODE: A Moving-Window-Based Framework for Detecting Concept Drift in Software Defect Prediction Symmetry software defect prediction cross-version defect prediction chronological-defect data class rebalancing concept drift |
| title | CODE: A Moving-Window-Based Framework for Detecting Concept Drift in Software Defect Prediction |
| title_full | CODE: A Moving-Window-Based Framework for Detecting Concept Drift in Software Defect Prediction |
| title_fullStr | CODE: A Moving-Window-Based Framework for Detecting Concept Drift in Software Defect Prediction |
| title_full_unstemmed | CODE: A Moving-Window-Based Framework for Detecting Concept Drift in Software Defect Prediction |
| title_short | CODE: A Moving-Window-Based Framework for Detecting Concept Drift in Software Defect Prediction |
| title_sort | code a moving window based framework for detecting concept drift in software defect prediction |
| topic | software defect prediction cross-version defect prediction chronological-defect data class rebalancing concept drift |
| url | https://www.mdpi.com/2073-8994/14/12/2508 |
| work_keys_str_mv | AT mdalamgirkabir codeamovingwindowbasedframeworkfordetectingconceptdriftinsoftwaredefectprediction AT shahinabegum codeamovingwindowbasedframeworkfordetectingconceptdriftinsoftwaredefectprediction AT mobyenuddinahmed codeamovingwindowbasedframeworkfordetectingconceptdriftinsoftwaredefectprediction AT atiqurrehman codeamovingwindowbasedframeworkfordetectingconceptdriftinsoftwaredefectprediction |