Adaptive Reward Computation in Reinforcement Learning-Based Continuous Integration Testing
Reinforcement learning (RL) has been applied to prioritizing test cases in Continuous Integration (CI) testing, where the reward plays a crucial role. It has been demonstrated that historical information-based reward function can improve the effectiveness of the test case prioritization (TCP). Howev...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2021-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/9366947/ |
| _version_ | 1798002054286802944 |
|---|---|
| author | Yang Yang Chaoyue Pan Zheng Li Ruilian Zhao |
| author_facet | Yang Yang Chaoyue Pan Zheng Li Ruilian Zhao |
| author_sort | Yang Yang |
| collection | DOAJ |
| description | Reinforcement learning (RL) has been applied to prioritizing test cases in Continuous Integration (CI) testing, where the reward plays a crucial role. It has been demonstrated that historical information-based reward function can improve the effectiveness of the test case prioritization (TCP). However, the inherent character of frequent iterations in CI can produce a considerable accumulation of historical information, which may decrease TCP efficiency and result in slow feedback. In this paper, the partial historical information is considered in the reward computation, where sliding window techniques are adopted to capture the possible efficient information. Firstly, the fixed-size sliding window is introduced to set a fixed length of recent historical information for each CI test. Then dynamic sliding window techniques are proposed, where the window size is continuously adaptive to each CI testing. Two methods are proposed, the test suite-based dynamic sliding window and the individual test case-based dynamic sliding window. The empirical studies are conducted on fourteen industrial-level programs, and the results reveal that under limited time, the sliding window-based reward function can effectively improve the TCP effect, where the NAPFD (Normalized Average Percentage of Faults Detected) and Recall of the dynamic sliding windows are better than that of the fixed-size sliding window. In particular, the individual test case-based dynamic sliding window approach can rank 74.18% failed test cases in the top 50% of the sorting sequence, with 1.35% improvement of NAPFD and 6.66 positions increased in TTF (Test to Fail). |
| first_indexed | 2024-04-11T11:46:04Z |
| format | Article |
| id | doaj.art-ef8fa91fe199494d94606e98d4485c48 |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-04-11T11:46:04Z |
| publishDate | 2021-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-ef8fa91fe199494d94606e98d4485c482022-12-22T04:25:35ZengIEEEIEEE Access2169-35362021-01-019366743668810.1109/ACCESS.2021.30632329366947Adaptive Reward Computation in Reinforcement Learning-Based Continuous Integration TestingYang Yang0https://orcid.org/0000-0001-9257-4631Chaoyue Pan1https://orcid.org/0000-0001-7877-7512Zheng Li2https://orcid.org/0000-0002-3938-7033Ruilian Zhao3https://orcid.org/0000-0002-6024-4010College of Information Science and Technology, Beijing University of Chemical Technology, Beijing, ChinaCollege of Information Science and Technology, Beijing University of Chemical Technology, Beijing, ChinaCollege of Information Science and Technology, Beijing University of Chemical Technology, Beijing, ChinaCollege of Information Science and Technology, Beijing University of Chemical Technology, Beijing, ChinaReinforcement learning (RL) has been applied to prioritizing test cases in Continuous Integration (CI) testing, where the reward plays a crucial role. It has been demonstrated that historical information-based reward function can improve the effectiveness of the test case prioritization (TCP). However, the inherent character of frequent iterations in CI can produce a considerable accumulation of historical information, which may decrease TCP efficiency and result in slow feedback. In this paper, the partial historical information is considered in the reward computation, where sliding window techniques are adopted to capture the possible efficient information. Firstly, the fixed-size sliding window is introduced to set a fixed length of recent historical information for each CI test. Then dynamic sliding window techniques are proposed, where the window size is continuously adaptive to each CI testing. Two methods are proposed, the test suite-based dynamic sliding window and the individual test case-based dynamic sliding window. The empirical studies are conducted on fourteen industrial-level programs, and the results reveal that under limited time, the sliding window-based reward function can effectively improve the TCP effect, where the NAPFD (Normalized Average Percentage of Faults Detected) and Recall of the dynamic sliding windows are better than that of the fixed-size sliding window. In particular, the individual test case-based dynamic sliding window approach can rank 74.18% failed test cases in the top 50% of the sorting sequence, with 1.35% improvement of NAPFD and 6.66 positions increased in TTF (Test to Fail).https://ieeexplore.ieee.org/document/9366947/Continuous integrationtest case prioritizationreinforcement learningsliding windowreward computation |
| spellingShingle | Yang Yang Chaoyue Pan Zheng Li Ruilian Zhao Adaptive Reward Computation in Reinforcement Learning-Based Continuous Integration Testing IEEE Access Continuous integration test case prioritization reinforcement learning sliding window reward computation |
| title | Adaptive Reward Computation in Reinforcement Learning-Based Continuous Integration Testing |
| title_full | Adaptive Reward Computation in Reinforcement Learning-Based Continuous Integration Testing |
| title_fullStr | Adaptive Reward Computation in Reinforcement Learning-Based Continuous Integration Testing |
| title_full_unstemmed | Adaptive Reward Computation in Reinforcement Learning-Based Continuous Integration Testing |
| title_short | Adaptive Reward Computation in Reinforcement Learning-Based Continuous Integration Testing |
| title_sort | adaptive reward computation in reinforcement learning based continuous integration testing |
| topic | Continuous integration test case prioritization reinforcement learning sliding window reward computation |
| url | https://ieeexplore.ieee.org/document/9366947/ |
| work_keys_str_mv | AT yangyang adaptiverewardcomputationinreinforcementlearningbasedcontinuousintegrationtesting AT chaoyuepan adaptiverewardcomputationinreinforcementlearningbasedcontinuousintegrationtesting AT zhengli adaptiverewardcomputationinreinforcementlearningbasedcontinuousintegrationtesting AT ruilianzhao adaptiverewardcomputationinreinforcementlearningbasedcontinuousintegrationtesting |