Making More with Less: Improving Software Testing Outcomes Using a Cross-Project and Cross-Language ML Classifier Based on Cost-Sensitive Training
As digitalization expands across all sectors, the economic toll of software defects on the U.S. economy reaches up to $2.41 trillion annually. High-profile incidents like the Boeing 787-Max 8 crash have shown the devastating potential of these defects, highlighting the critical importance of softwar...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Published: |
MDPI AG
2024
|
| Online Access: | https://hdl.handle.net/1721.1/155268 |
| _version_ | 1811069783031939072 |
|---|---|
| author | Nascimento, Alexandre M. Shimanuki, Gabriel Kenji G. Dias, Luiz Alberto V. |
| author_facet | Nascimento, Alexandre M. Shimanuki, Gabriel Kenji G. Dias, Luiz Alberto V. |
| author_sort | Nascimento, Alexandre M. |
| collection | MIT |
| description | As digitalization expands across all sectors, the economic toll of software defects on the U.S. economy reaches up to $2.41 trillion annually. High-profile incidents like the Boeing 787-Max 8 crash have shown the devastating potential of these defects, highlighting the critical importance of software testing within quality assurance frameworks. However, due to its complexity and resource intensity, the exhaustive nature of comprehensive testing often surpasses budget constraints. This research utilizes a machine learning (ML) model to enhance software testing decisions by pinpointing areas most susceptible to defects and optimizing scarce resource allocation. Previous studies have shown promising results using cost-sensitive training to refine ML models, improving predictive accuracy by reducing false negatives through addressing class imbalances in defect prediction datasets. This approach facilitates more targeted and effective testing efforts. Nevertheless, these models’ in-company generalizability across different projects (cross-project) and programming languages (cross-language) remained untested. This study validates the approach’s applicability across diverse development environments by integrating various datasets from distinct projects into a unified dataset, using a more interpretable ML technique. The results demonstrate that ML can support software testing decisions, enabling teams to identify up to 7× more defective modules compared to benchmark with the same testing effort. |
| first_indexed | 2024-09-23T08:16:03Z |
| format | Article |
| id | mit-1721.1/155268 |
| institution | Massachusetts Institute of Technology |
| last_indexed | 2024-09-23T08:16:03Z |
| publishDate | 2024 |
| publisher | MDPI AG |
| record_format | dspace |
| spelling | mit-1721.1/1552682024-06-14T05:43:10Z Making More with Less: Improving Software Testing Outcomes Using a Cross-Project and Cross-Language ML Classifier Based on Cost-Sensitive Training Nascimento, Alexandre M. Shimanuki, Gabriel Kenji G. Dias, Luiz Alberto V. As digitalization expands across all sectors, the economic toll of software defects on the U.S. economy reaches up to $2.41 trillion annually. High-profile incidents like the Boeing 787-Max 8 crash have shown the devastating potential of these defects, highlighting the critical importance of software testing within quality assurance frameworks. However, due to its complexity and resource intensity, the exhaustive nature of comprehensive testing often surpasses budget constraints. This research utilizes a machine learning (ML) model to enhance software testing decisions by pinpointing areas most susceptible to defects and optimizing scarce resource allocation. Previous studies have shown promising results using cost-sensitive training to refine ML models, improving predictive accuracy by reducing false negatives through addressing class imbalances in defect prediction datasets. This approach facilitates more targeted and effective testing efforts. Nevertheless, these models’ in-company generalizability across different projects (cross-project) and programming languages (cross-language) remained untested. This study validates the approach’s applicability across diverse development environments by integrating various datasets from distinct projects into a unified dataset, using a more interpretable ML technique. The results demonstrate that ML can support software testing decisions, enabling teams to identify up to 7× more defective modules compared to benchmark with the same testing effort. 2024-06-13T18:32:13Z 2024-06-13T18:32:13Z 2024-06-04 2024-06-13T14:54:19Z Article http://purl.org/eprint/type/JournalArticle 2076-3417 https://hdl.handle.net/1721.1/155268 Nascimento, A.M.; Shimanuki, G.K.G.; Dias, L.A.V. Making More with Less: Improving Software Testing Outcomes Using a Cross-Project and Cross-Language ML Classifier Based on Cost-Sensitive Training. Appl. Sci. 2024, 14, 4880. PUBLISHER_CC 10.3390/app14114880 Applied Sciences Creative Commons Attribution https://creativecommons.org/licenses/by/4.0/ application/pdf MDPI AG Multidisciplinary Digital Publishing Institute |
| spellingShingle | Nascimento, Alexandre M. Shimanuki, Gabriel Kenji G. Dias, Luiz Alberto V. Making More with Less: Improving Software Testing Outcomes Using a Cross-Project and Cross-Language ML Classifier Based on Cost-Sensitive Training |
| title | Making More with Less: Improving Software Testing Outcomes Using a Cross-Project and Cross-Language ML Classifier Based on Cost-Sensitive Training |
| title_full | Making More with Less: Improving Software Testing Outcomes Using a Cross-Project and Cross-Language ML Classifier Based on Cost-Sensitive Training |
| title_fullStr | Making More with Less: Improving Software Testing Outcomes Using a Cross-Project and Cross-Language ML Classifier Based on Cost-Sensitive Training |
| title_full_unstemmed | Making More with Less: Improving Software Testing Outcomes Using a Cross-Project and Cross-Language ML Classifier Based on Cost-Sensitive Training |
| title_short | Making More with Less: Improving Software Testing Outcomes Using a Cross-Project and Cross-Language ML Classifier Based on Cost-Sensitive Training |
| title_sort | making more with less improving software testing outcomes using a cross project and cross language ml classifier based on cost sensitive training |
| url | https://hdl.handle.net/1721.1/155268 |
| work_keys_str_mv | AT nascimentoalexandrem makingmorewithlessimprovingsoftwaretestingoutcomesusingacrossprojectandcrosslanguagemlclassifierbasedoncostsensitivetraining AT shimanukigabrielkenjig makingmorewithlessimprovingsoftwaretestingoutcomesusingacrossprojectandcrosslanguagemlclassifierbasedoncostsensitivetraining AT diasluizalbertov makingmorewithlessimprovingsoftwaretestingoutcomesusingacrossprojectandcrosslanguagemlclassifierbasedoncostsensitivetraining |