An Empirical Study of Greedy Test Suite Minimization Techniques Using Mutation Coverage
Test suite minimization is the task of finding a smaller test suite that still fulfills the properties of the original test suite but which comprises fewer test cases. It is important in practice, especially in the context of regression testing, where test suites are re-executed. However, test suite...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2023-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10160008/ |
| _version_ | 1797787382439739392 |
|---|---|
| author | Seema Jehan Franz Wotawa |
| author_facet | Seema Jehan Franz Wotawa |
| author_sort | Seema Jehan |
| collection | DOAJ |
| description | Test suite minimization is the task of finding a smaller test suite that still fulfills the properties of the original test suite but which comprises fewer test cases. It is important in practice, especially in the context of regression testing, where test suites are re-executed. However, test suite minimization as a set covering problem is known as an NP-complete problem, which requires applications of heuristics. Although many test suite minimization techniques have been applied previously but obtained conflicting results primarily due to inherent differences in underlying programming languages and experimental setup. In this respect, we study traditional greedy-based algorithms for test suite minimization that allow to remove test cases in a way such that the reduced test suite satisfies all requirements. Specifically, we evaluated commonly discussed approaches on publicly available JavaScript applications using mutation coverage. We show that the discussed algorithms reduce the test suite size of the studied example programs on average to 70% without compromising the fault-detection capability of the original test suite. The suggested approach not only minimizes the test suite’s size, thereby reducing the regression testing cost, but also ensures that the reduced test suite catches the same number of faults as that of the original test suite. Further, we also examine their performance in scenarios when meeting all testing requirements is not feasible due to time and budget constraints. |
| first_indexed | 2024-03-13T01:21:08Z |
| format | Article |
| id | doaj.art-2c476261156943f588292e22db8300bd |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-03-13T01:21:08Z |
| publishDate | 2023-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-2c476261156943f588292e22db8300bd2023-07-04T23:00:21ZengIEEEIEEE Access2169-35362023-01-0111654276544210.1109/ACCESS.2023.328907310160008An Empirical Study of Greedy Test Suite Minimization Techniques Using Mutation CoverageSeema Jehan0https://orcid.org/0000-0001-6379-6972Franz Wotawa1https://orcid.org/0000-0002-0462-2283School of Electrical Engineering and Computer Science, National University of Sciences and Technology, Islamabad, PakistanChristian Doppler Laboratory for Quality Assurance Methods for Autonomous Cyber-Physical Systems, Institute for Software Technology, Graz of Technical University, Graz, AustriaTest suite minimization is the task of finding a smaller test suite that still fulfills the properties of the original test suite but which comprises fewer test cases. It is important in practice, especially in the context of regression testing, where test suites are re-executed. However, test suite minimization as a set covering problem is known as an NP-complete problem, which requires applications of heuristics. Although many test suite minimization techniques have been applied previously but obtained conflicting results primarily due to inherent differences in underlying programming languages and experimental setup. In this respect, we study traditional greedy-based algorithms for test suite minimization that allow to remove test cases in a way such that the reduced test suite satisfies all requirements. Specifically, we evaluated commonly discussed approaches on publicly available JavaScript applications using mutation coverage. We show that the discussed algorithms reduce the test suite size of the studied example programs on average to 70% without compromising the fault-detection capability of the original test suite. The suggested approach not only minimizes the test suite’s size, thereby reducing the regression testing cost, but also ensures that the reduced test suite catches the same number of faults as that of the original test suite. Further, we also examine their performance in scenarios when meeting all testing requirements is not feasible due to time and budget constraints.https://ieeexplore.ieee.org/document/10160008/Test suite minimizationmutation testingregression testingJavaScript |
| spellingShingle | Seema Jehan Franz Wotawa An Empirical Study of Greedy Test Suite Minimization Techniques Using Mutation Coverage IEEE Access Test suite minimization mutation testing regression testing JavaScript |
| title | An Empirical Study of Greedy Test Suite Minimization Techniques Using Mutation Coverage |
| title_full | An Empirical Study of Greedy Test Suite Minimization Techniques Using Mutation Coverage |
| title_fullStr | An Empirical Study of Greedy Test Suite Minimization Techniques Using Mutation Coverage |
| title_full_unstemmed | An Empirical Study of Greedy Test Suite Minimization Techniques Using Mutation Coverage |
| title_short | An Empirical Study of Greedy Test Suite Minimization Techniques Using Mutation Coverage |
| title_sort | empirical study of greedy test suite minimization techniques using mutation coverage |
| topic | Test suite minimization mutation testing regression testing JavaScript |
| url | https://ieeexplore.ieee.org/document/10160008/ |
| work_keys_str_mv | AT seemajehan anempiricalstudyofgreedytestsuiteminimizationtechniquesusingmutationcoverage AT franzwotawa anempiricalstudyofgreedytestsuiteminimizationtechniquesusingmutationcoverage AT seemajehan empiricalstudyofgreedytestsuiteminimizationtechniquesusingmutationcoverage AT franzwotawa empiricalstudyofgreedytestsuiteminimizationtechniquesusingmutationcoverage |