Measuring the developmental function of peer review: a multi-dimensional, cross-disciplinary analysis of peer review reports from 740 academic journals
Reviewers do not only help editors to screen manuscripts for publication in academic journals; they also serve to increase the rigor and value of manuscripts by constructive feedback. However, measuring this developmental function of peer review is difficult as it requires fine-grained data on repor...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
PeerJ Inc.
2022-06-01
|
| Series: | PeerJ |
| Subjects: | |
| Online Access: | https://peerj.com/articles/13539.pdf |
| _version_ | 1797425917729964032 |
|---|---|
| author | Daniel Garcia-Costa Flaminio Squazzoni Bahar Mehmani Francisco Grimaldo |
| author_facet | Daniel Garcia-Costa Flaminio Squazzoni Bahar Mehmani Francisco Grimaldo |
| author_sort | Daniel Garcia-Costa |
| collection | DOAJ |
| description | Reviewers do not only help editors to screen manuscripts for publication in academic journals; they also serve to increase the rigor and value of manuscripts by constructive feedback. However, measuring this developmental function of peer review is difficult as it requires fine-grained data on reports and journals without any optimal benchmark. To fill this gap, we adapted a recently proposed quality assessment tool and tested it on a sample of 1.3 million reports submitted to 740 Elsevier journals in 2018–2020. Results showed that the developmental standards of peer review are shared across areas of research, yet with remarkable differences. Reports submitted to social science and economics journals show the highest developmental standards. Reports from junior reviewers, women and reviewers from Western Europe are generally more developmental than those from senior, men and reviewers working in academic institutions outside Western regions. Our findings suggest that increasing the standards of peer review at journals requires effort to assess interventions and measure practices with context-specific and multi-dimensional frameworks. |
| first_indexed | 2024-03-09T08:21:56Z |
| format | Article |
| id | doaj.art-a5d6254823ef452dbb9711cf80774571 |
| institution | Directory Open Access Journal |
| issn | 2167-8359 |
| language | English |
| last_indexed | 2024-03-09T08:21:56Z |
| publishDate | 2022-06-01 |
| publisher | PeerJ Inc. |
| record_format | Article |
| series | PeerJ |
| spelling | doaj.art-a5d6254823ef452dbb9711cf807745712023-12-02T21:42:37ZengPeerJ Inc.PeerJ2167-83592022-06-0110e1353910.7717/peerj.13539Measuring the developmental function of peer review: a multi-dimensional, cross-disciplinary analysis of peer review reports from 740 academic journalsDaniel Garcia-Costa0Flaminio Squazzoni1Bahar Mehmani2Francisco Grimaldo3Department of Computer Science, University of Valencia, Valencia, SpainDepartment of Social and Political Sciences, University of Milan, Milan, Lombardy, ItalySTM Journals, Elsevier, Amsterdam, The NetherlandsDepartment of Computer Science, University of Valencia, Valencia, SpainReviewers do not only help editors to screen manuscripts for publication in academic journals; they also serve to increase the rigor and value of manuscripts by constructive feedback. However, measuring this developmental function of peer review is difficult as it requires fine-grained data on reports and journals without any optimal benchmark. To fill this gap, we adapted a recently proposed quality assessment tool and tested it on a sample of 1.3 million reports submitted to 740 Elsevier journals in 2018–2020. Results showed that the developmental standards of peer review are shared across areas of research, yet with remarkable differences. Reports submitted to social science and economics journals show the highest developmental standards. Reports from junior reviewers, women and reviewers from Western Europe are generally more developmental than those from senior, men and reviewers working in academic institutions outside Western regions. Our findings suggest that increasing the standards of peer review at journals requires effort to assess interventions and measure practices with context-specific and multi-dimensional frameworks.https://peerj.com/articles/13539.pdfPeer reviewStandardsReviewersAcademic journalsNatural language processing |
| spellingShingle | Daniel Garcia-Costa Flaminio Squazzoni Bahar Mehmani Francisco Grimaldo Measuring the developmental function of peer review: a multi-dimensional, cross-disciplinary analysis of peer review reports from 740 academic journals PeerJ Peer review Standards Reviewers Academic journals Natural language processing |
| title | Measuring the developmental function of peer review: a multi-dimensional, cross-disciplinary analysis of peer review reports from 740 academic journals |
| title_full | Measuring the developmental function of peer review: a multi-dimensional, cross-disciplinary analysis of peer review reports from 740 academic journals |
| title_fullStr | Measuring the developmental function of peer review: a multi-dimensional, cross-disciplinary analysis of peer review reports from 740 academic journals |
| title_full_unstemmed | Measuring the developmental function of peer review: a multi-dimensional, cross-disciplinary analysis of peer review reports from 740 academic journals |
| title_short | Measuring the developmental function of peer review: a multi-dimensional, cross-disciplinary analysis of peer review reports from 740 academic journals |
| title_sort | measuring the developmental function of peer review a multi dimensional cross disciplinary analysis of peer review reports from 740 academic journals |
| topic | Peer review Standards Reviewers Academic journals Natural language processing |
| url | https://peerj.com/articles/13539.pdf |
| work_keys_str_mv | AT danielgarciacosta measuringthedevelopmentalfunctionofpeerreviewamultidimensionalcrossdisciplinaryanalysisofpeerreviewreportsfrom740academicjournals AT flaminiosquazzoni measuringthedevelopmentalfunctionofpeerreviewamultidimensionalcrossdisciplinaryanalysisofpeerreviewreportsfrom740academicjournals AT baharmehmani measuringthedevelopmentalfunctionofpeerreviewamultidimensionalcrossdisciplinaryanalysisofpeerreviewreportsfrom740academicjournals AT franciscogrimaldo measuringthedevelopmentalfunctionofpeerreviewamultidimensionalcrossdisciplinaryanalysisofpeerreviewreportsfrom740academicjournals |