Generation and application of constrained interaction test suites using base forbidden tuples with a mixed neighborhood tabu search
To ensure the quality of current highly configurable software systems, intensive testing is needed to test all the configuration combinations and detect all the possible faults. This task becomes more challenging for most modern software systems when constraints are given for the configurations. Her...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
World Scientific Publishing Co. Pte Ltd
2020
|
| Subjects: | |
| Online Access: | http://umpir.ump.edu.my/id/eprint/31015/1/Generation%20and%20application%20of%20constrained%20interaction%20test%20suites%20.pdf |
| _version_ | 1825813786874871808 |
|---|---|
| author | Hasan, Imad H. Ahmed, Bestoun S. Potrus, Moayad Y. Kamal Z., Zamli |
| author_facet | Hasan, Imad H. Ahmed, Bestoun S. Potrus, Moayad Y. Kamal Z., Zamli |
| author_sort | Hasan, Imad H. |
| collection | UMP |
| description | To ensure the quality of current highly configurable software systems, intensive testing is needed to test all the configuration combinations and detect all the possible faults. This task becomes more challenging for most modern software systems when constraints are given for the configurations. Here, intensive testing is almost impossible, especially considering the additional computation required to resolve the constraints during the test generation process. In addition, this testing process is exhaustive and time-consuming. Combinatorial interaction strategies can systematically reduce the number of test cases to construct a minimal test suite without affecting the effectiveness of the tests. This paper presents a new efficient search-based strategy to generate constrained interaction test suites to cover all possible combinations. The paper also shows a new application of constrained interaction testing in software fault searches. The proposed strategy initially generates the set of all possible tt-tupletuple combinations; then, it filters out the set by removing the forbidden tt-tuplestuples using the Base Forbidden Tuple (BFT) approach. The strategy also utilizes a mixed neighborhood tabu search (TS) to construct optimal or near-optimal constrained test suites. The efficiency of the proposed method is evaluated through a comparison against two well-known state-of-the-art tools. The evaluation consists of three sets of experiments for 35 standard benchmarks. Additionally, the effectiveness and quality of the results are assessed using a real-world case study. Experimental results show that the proposed strategy outperforms one of the competitive strategies, ACTS, for approximately 83% of the benchmarks and achieves similar results to CASA for 65% of the benchmarks when the interaction strength is 2. For an interaction strength of 3, the proposed method outperforms other competitive strategies for approximately 60% and 42% of the benchmarks. The proposed strategy can also generate constrained interaction test suites for an interaction strength of 4, which is not possible for many strategies. The real-world case study shows that the generated test suites can effectively detect injected faults using mutation testing. |
| first_indexed | 2024-03-06T12:49:12Z |
| format | Article |
| id | UMPir31015 |
| institution | Universiti Malaysia Pahang |
| language | English |
| last_indexed | 2024-03-06T12:49:12Z |
| publishDate | 2020 |
| publisher | World Scientific Publishing Co. Pte Ltd |
| record_format | dspace |
| spelling | UMPir310152021-06-30T08:24:45Z http://umpir.ump.edu.my/id/eprint/31015/ Generation and application of constrained interaction test suites using base forbidden tuples with a mixed neighborhood tabu search Hasan, Imad H. Ahmed, Bestoun S. Potrus, Moayad Y. Kamal Z., Zamli QA76 Computer software To ensure the quality of current highly configurable software systems, intensive testing is needed to test all the configuration combinations and detect all the possible faults. This task becomes more challenging for most modern software systems when constraints are given for the configurations. Here, intensive testing is almost impossible, especially considering the additional computation required to resolve the constraints during the test generation process. In addition, this testing process is exhaustive and time-consuming. Combinatorial interaction strategies can systematically reduce the number of test cases to construct a minimal test suite without affecting the effectiveness of the tests. This paper presents a new efficient search-based strategy to generate constrained interaction test suites to cover all possible combinations. The paper also shows a new application of constrained interaction testing in software fault searches. The proposed strategy initially generates the set of all possible tt-tupletuple combinations; then, it filters out the set by removing the forbidden tt-tuplestuples using the Base Forbidden Tuple (BFT) approach. The strategy also utilizes a mixed neighborhood tabu search (TS) to construct optimal or near-optimal constrained test suites. The efficiency of the proposed method is evaluated through a comparison against two well-known state-of-the-art tools. The evaluation consists of three sets of experiments for 35 standard benchmarks. Additionally, the effectiveness and quality of the results are assessed using a real-world case study. Experimental results show that the proposed strategy outperforms one of the competitive strategies, ACTS, for approximately 83% of the benchmarks and achieves similar results to CASA for 65% of the benchmarks when the interaction strength is 2. For an interaction strength of 3, the proposed method outperforms other competitive strategies for approximately 60% and 42% of the benchmarks. The proposed strategy can also generate constrained interaction test suites for an interaction strength of 4, which is not possible for many strategies. The real-world case study shows that the generated test suites can effectively detect injected faults using mutation testing. World Scientific Publishing Co. Pte Ltd 2020-03-01 Article PeerReviewed pdf en http://umpir.ump.edu.my/id/eprint/31015/1/Generation%20and%20application%20of%20constrained%20interaction%20test%20suites%20.pdf Hasan, Imad H. and Ahmed, Bestoun S. and Potrus, Moayad Y. and Kamal Z., Zamli (2020) Generation and application of constrained interaction test suites using base forbidden tuples with a mixed neighborhood tabu search. International Journal of Software Engineering and Knowledge Engineering, 30 (1). pp. 363-398. ISSN 0218-1940. (Published) https://doi.org/10.1142/S0218194020500151 https://doi.org/10.1142/S0218194020500151 |
| spellingShingle | QA76 Computer software Hasan, Imad H. Ahmed, Bestoun S. Potrus, Moayad Y. Kamal Z., Zamli Generation and application of constrained interaction test suites using base forbidden tuples with a mixed neighborhood tabu search |
| title | Generation and application of constrained interaction test suites using base forbidden tuples with a mixed neighborhood tabu search |
| title_full | Generation and application of constrained interaction test suites using base forbidden tuples with a mixed neighborhood tabu search |
| title_fullStr | Generation and application of constrained interaction test suites using base forbidden tuples with a mixed neighborhood tabu search |
| title_full_unstemmed | Generation and application of constrained interaction test suites using base forbidden tuples with a mixed neighborhood tabu search |
| title_short | Generation and application of constrained interaction test suites using base forbidden tuples with a mixed neighborhood tabu search |
| title_sort | generation and application of constrained interaction test suites using base forbidden tuples with a mixed neighborhood tabu search |
| topic | QA76 Computer software |
| url | http://umpir.ump.edu.my/id/eprint/31015/1/Generation%20and%20application%20of%20constrained%20interaction%20test%20suites%20.pdf |
| work_keys_str_mv | AT hasanimadh generationandapplicationofconstrainedinteractiontestsuitesusingbaseforbiddentupleswithamixedneighborhoodtabusearch AT ahmedbestouns generationandapplicationofconstrainedinteractiontestsuitesusingbaseforbiddentupleswithamixedneighborhoodtabusearch AT potrusmoayady generationandapplicationofconstrainedinteractiontestsuitesusingbaseforbiddentupleswithamixedneighborhoodtabusearch AT kamalzzamli generationandapplicationofconstrainedinteractiontestsuitesusingbaseforbiddentupleswithamixedneighborhoodtabusearch |