Generated Tests in the Context of Maintenance Tasks: A Series of Empirical Studies
Maintenance tasks are often time-consuming and hard to manage. To cope with that, developers often use failing test cases to guide their maintenance efforts. Therefore, working with good test cases is essential to the success of maintenance. Automatically generated tests can save time and lead to hi...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2022-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/9954000/ |
| _version_ | 1797986755631120384 |
|---|---|
| author | Wesley B. R. Herculano Everton L. G. Alves Melina Mongiovi |
| author_facet | Wesley B. R. Herculano Everton L. G. Alves Melina Mongiovi |
| author_sort | Wesley B. R. Herculano |
| collection | DOAJ |
| description | Maintenance tasks are often time-consuming and hard to manage. To cope with that, developers often use failing test cases to guide their maintenance efforts. Therefore, working with good test cases is essential to the success of maintenance. Automatically generated tests can save time and lead to higher code coverage. However, they often include test smells and are hard to read. Moreover, it is not clear whether generated tests can be effective when guiding maintenance tasks, nor if developers fully accept them. This paper presents a series of empirical studies that evaluates whether automatically generated tests can support developers when maintaining code. First, we ran an empirical study with 20 real developers to compare how they perform maintenance tasks with automatically generated (Evosuite or Randoop) and manually-written tests. Then, we applied a survey with 82 developers that assessed developers’ perceptions of using refactored Randoop tests. Finally, a third empirical study with 24 developers focused on evaluating the impact of refactored Randoop tests on the performance of maintenance. Our results and statistical analyses show that automatically generated tests can be a great help for identifying faults during maintenance. Developers were more accurate at identifying maintenance tasks when using Evosuite tests and equally effective to create bug fixes (manually-written, Evosuite, and Randoop). Developers preferred refactored Randoop tests to the original ones. However, the refactorings applied did not improve their performance on fault detection. On the other hand, developers were more effective to fix the faults using refactored Randoop tests. |
| first_indexed | 2024-04-11T07:37:54Z |
| format | Article |
| id | doaj.art-dd5f93e72ffc4ec18712ced70223f802 |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-04-11T07:37:54Z |
| publishDate | 2022-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-dd5f93e72ffc4ec18712ced70223f8022022-12-22T04:36:41ZengIEEEIEEE Access2169-35362022-01-011012141812144310.1109/ACCESS.2022.32228039954000Generated Tests in the Context of Maintenance Tasks: A Series of Empirical StudiesWesley B. R. Herculano0https://orcid.org/0000-0002-5403-3903Everton L. G. Alves1Melina Mongiovi2Department of Systems and Computing, Federal University of Campina Grande, Campina Grande, BrazilDepartment of Systems and Computing, Federal University of Campina Grande, Campina Grande, BrazilDepartment of Systems and Computing, Federal University of Campina Grande, Campina Grande, BrazilMaintenance tasks are often time-consuming and hard to manage. To cope with that, developers often use failing test cases to guide their maintenance efforts. Therefore, working with good test cases is essential to the success of maintenance. Automatically generated tests can save time and lead to higher code coverage. However, they often include test smells and are hard to read. Moreover, it is not clear whether generated tests can be effective when guiding maintenance tasks, nor if developers fully accept them. This paper presents a series of empirical studies that evaluates whether automatically generated tests can support developers when maintaining code. First, we ran an empirical study with 20 real developers to compare how they perform maintenance tasks with automatically generated (Evosuite or Randoop) and manually-written tests. Then, we applied a survey with 82 developers that assessed developers’ perceptions of using refactored Randoop tests. Finally, a third empirical study with 24 developers focused on evaluating the impact of refactored Randoop tests on the performance of maintenance. Our results and statistical analyses show that automatically generated tests can be a great help for identifying faults during maintenance. Developers were more accurate at identifying maintenance tasks when using Evosuite tests and equally effective to create bug fixes (manually-written, Evosuite, and Randoop). Developers preferred refactored Randoop tests to the original ones. However, the refactorings applied did not improve their performance on fault detection. On the other hand, developers were more effective to fix the faults using refactored Randoop tests.https://ieeexplore.ieee.org/document/9954000/Generated testsmaintenancerandoopevosuitetest smellsrefactoring |
| spellingShingle | Wesley B. R. Herculano Everton L. G. Alves Melina Mongiovi Generated Tests in the Context of Maintenance Tasks: A Series of Empirical Studies IEEE Access Generated tests maintenance randoop evosuite test smells refactoring |
| title | Generated Tests in the Context of Maintenance Tasks: A Series of Empirical Studies |
| title_full | Generated Tests in the Context of Maintenance Tasks: A Series of Empirical Studies |
| title_fullStr | Generated Tests in the Context of Maintenance Tasks: A Series of Empirical Studies |
| title_full_unstemmed | Generated Tests in the Context of Maintenance Tasks: A Series of Empirical Studies |
| title_short | Generated Tests in the Context of Maintenance Tasks: A Series of Empirical Studies |
| title_sort | generated tests in the context of maintenance tasks a series of empirical studies |
| topic | Generated tests maintenance randoop evosuite test smells refactoring |
| url | https://ieeexplore.ieee.org/document/9954000/ |
| work_keys_str_mv | AT wesleybrherculano generatedtestsinthecontextofmaintenancetasksaseriesofempiricalstudies AT evertonlgalves generatedtestsinthecontextofmaintenancetasksaseriesofempiricalstudies AT melinamongiovi generatedtestsinthecontextofmaintenancetasksaseriesofempiricalstudies |