Helping Pull Request Reviewer Recommendation Systems to Focus
The selection of code reviewers for a pull request can impact the quality as well as the speed of the review. In order to have the best experience both for the developer and the reviewers, there is a need for automatic reviewer recommendation systems for pull requests. Although there has been previo...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2023-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10172184/ |
| _version_ | 1797617759505350656 |
|---|---|
| author | Nikola Pejic Zaharije Radivojevic Milos Cvetanovic |
| author_facet | Nikola Pejic Zaharije Radivojevic Milos Cvetanovic |
| author_sort | Nikola Pejic |
| collection | DOAJ |
| description | The selection of code reviewers for a pull request can impact the quality as well as the speed of the review. In order to have the best experience both for the developer and the reviewers, there is a need for automatic reviewer recommendation systems for pull requests. Although there has been previous research in this area, it was mainly focused on smaller and medium repositories of up to around 200 developers, while larger repositories were rarely targeted. In this paper we evaluate several existing approaches on a set of 8 Microsoft repositories of different sizes, noticing that the average performance of the approaches seemed to decrease with the number of reviewers the repository has. In order to focus the existing approaches only on relevant reviewers, we propose a technique for improving their performance by scoping down the set of candidate reviewers based on multiple filters. We defined several basic filters and determined that 5 out of the 7 tested existing approaches experienced performance improvements of up to 16.24% better precision and 19.66% better recall averaged over all datasets, with the per dataset improvements peaking at 36.63% better precision and 28.63% better recall. Additionally, by combining different basic filters we were able to achieve additional improvements for 6 out of the 7 existing approaches (with a minor improvement for the remaining approach), which on average over all datasets had up to 17.60% better precision and 21.23% better recall, while the per dataset improvements peaked at 43.28% better precision and 30.94% better recall. |
| first_indexed | 2024-03-11T07:59:20Z |
| format | Article |
| id | doaj.art-ff9ae0e484414985b037d0ecdebaef57 |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-03-11T07:59:20Z |
| publishDate | 2023-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-ff9ae0e484414985b037d0ecdebaef572023-11-17T00:01:12ZengIEEEIEEE Access2169-35362023-01-0111710137102510.1109/ACCESS.2023.329205610172184Helping Pull Request Reviewer Recommendation Systems to FocusNikola Pejic0https://orcid.org/0000-0003-4669-3642Zaharije Radivojevic1Milos Cvetanovic2https://orcid.org/0000-0003-1063-0340School of Electrical Engineering, University of Belgrade, Belgrade, SerbiaMicrosoft Development Center Serbia, Belgrade, SerbiaSchool of Electrical Engineering, University of Belgrade, Belgrade, SerbiaThe selection of code reviewers for a pull request can impact the quality as well as the speed of the review. In order to have the best experience both for the developer and the reviewers, there is a need for automatic reviewer recommendation systems for pull requests. Although there has been previous research in this area, it was mainly focused on smaller and medium repositories of up to around 200 developers, while larger repositories were rarely targeted. In this paper we evaluate several existing approaches on a set of 8 Microsoft repositories of different sizes, noticing that the average performance of the approaches seemed to decrease with the number of reviewers the repository has. In order to focus the existing approaches only on relevant reviewers, we propose a technique for improving their performance by scoping down the set of candidate reviewers based on multiple filters. We defined several basic filters and determined that 5 out of the 7 tested existing approaches experienced performance improvements of up to 16.24% better precision and 19.66% better recall averaged over all datasets, with the per dataset improvements peaking at 36.63% better precision and 28.63% better recall. Additionally, by combining different basic filters we were able to achieve additional improvements for 6 out of the 7 existing approaches (with a minor improvement for the remaining approach), which on average over all datasets had up to 17.60% better precision and 21.23% better recall, while the per dataset improvements peaked at 43.28% better precision and 30.94% better recall.https://ieeexplore.ieee.org/document/10172184/Modern code reviewpull requestreviewer recommendation |
| spellingShingle | Nikola Pejic Zaharije Radivojevic Milos Cvetanovic Helping Pull Request Reviewer Recommendation Systems to Focus IEEE Access Modern code review pull request reviewer recommendation |
| title | Helping Pull Request Reviewer Recommendation Systems to Focus |
| title_full | Helping Pull Request Reviewer Recommendation Systems to Focus |
| title_fullStr | Helping Pull Request Reviewer Recommendation Systems to Focus |
| title_full_unstemmed | Helping Pull Request Reviewer Recommendation Systems to Focus |
| title_short | Helping Pull Request Reviewer Recommendation Systems to Focus |
| title_sort | helping pull request reviewer recommendation systems to focus |
| topic | Modern code review pull request reviewer recommendation |
| url | https://ieeexplore.ieee.org/document/10172184/ |
| work_keys_str_mv | AT nikolapejic helpingpullrequestreviewerrecommendationsystemstofocus AT zaharijeradivojevic helpingpullrequestreviewerrecommendationsystemstofocus AT miloscvetanovic helpingpullrequestreviewerrecommendationsystemstofocus |