Peer reviewers' willingness to review, their recommendations and quality of reviews after the Finnish Medical Journal switched from single-blind to double-blind peer review
Abstract Background There is a power imbalance between authors and reviewers in single-blind peer review. We explored how switching from single-blind to double-blind peer review affected 1) the willingness of experts to review, 2) their publication recommendations, and 3) the quality of review repor...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
BMC
2023-10-01
|
| Series: | Research Integrity and Peer Review |
| Subjects: | |
| Online Access: | https://doi.org/10.1186/s41073-023-00140-6 |
| _version_ | 1797555738931888128 |
|---|---|
| author | Piitu Parmanne Joonas Laajava Noora Järvinen Terttu Harju Mauri Marttunen Pertti Saloheimo |
| author_facet | Piitu Parmanne Joonas Laajava Noora Järvinen Terttu Harju Mauri Marttunen Pertti Saloheimo |
| author_sort | Piitu Parmanne |
| collection | DOAJ |
| description | Abstract Background There is a power imbalance between authors and reviewers in single-blind peer review. We explored how switching from single-blind to double-blind peer review affected 1) the willingness of experts to review, 2) their publication recommendations, and 3) the quality of review reports. Methods The Finnish Medical Journal switched from single-blind to double-blind peer review in September 2017. The proportion of review invitations that resulted in a received review report was counted. The reviewers’ recommendations of “accept as is”, “minor revision”, “major revision” or “reject” were explored. The content of the reviews was assessed by two experienced reviewers using the Review Quality Instrument modified to apply to both original research and review manuscripts. The study material comprised reviews submitted from September 2017 to February 2018. The controls were the reviews submitted between September 2015 and February 2016 and between September 2016 and February 2017. The reviewers’ recommendations and the scorings of quality assessments were tested with the Chi square test, and the means of quality assessments with the independent-samples t test. Results A total of 118 double-blind first-round reviews of 59 manuscripts were compared with 232 single-blind first-round reviews of 116 manuscripts. The proportion of successful review invitations when reviewing single-blinded was 67%, and when reviewing double-blinded, 66%. When reviewing double-blinded, the reviewers recommended accept as is or minor revision less often than during the control period (59% vs. 73%), and major revision or rejection more often (41% vs 27%, P = 0.010). For the quality assessment, 116 reviews from the double-blind period were compared with 104 reviews conducted between September 2016 and February 2017. On a 1–5 scale (1 poor, 5 excellent), double-blind reviews received higher overall proportion of ratings of 4 and 5 than single-blind reviews (56% vs. 49%, P < 0.001). Means for the overall quality of double-blind reviews were 3.38 (IQR, 3.33–3.44) vs. 3.22 (3.17–3.28; P < 0.001) for single-blind reviews. Conclusions The quality of the reviews conducted double-blind was better than of those conducted single-blind. Switching to double-blind review did not alter the reviewers’ willingness to review. The reviewers became slightly more critical. |
| first_indexed | 2024-03-10T16:51:45Z |
| format | Article |
| id | doaj.art-5ffb8f1c265241cb9cf3b85587c73ed3 |
| institution | Directory Open Access Journal |
| issn | 2058-8615 |
| language | English |
| last_indexed | 2024-03-10T16:51:45Z |
| publishDate | 2023-10-01 |
| publisher | BMC |
| record_format | Article |
| series | Research Integrity and Peer Review |
| spelling | doaj.art-5ffb8f1c265241cb9cf3b85587c73ed32023-11-20T11:16:41ZengBMCResearch Integrity and Peer Review2058-86152023-10-01811710.1186/s41073-023-00140-6Peer reviewers' willingness to review, their recommendations and quality of reviews after the Finnish Medical Journal switched from single-blind to double-blind peer reviewPiitu Parmanne0Joonas Laajava1Noora Järvinen2Terttu Harju3Mauri Marttunen4Pertti Saloheimo5Finnish Medical AssociationUniversity of HelsinkiFinnish Medical JournalRespiratory Medicine Research Unit, Department of Medicine, University of OuluDepartment of Adolescent Psychiatry, Helsinki University Hospital and University of HelsinkiFinnish Medical JournalAbstract Background There is a power imbalance between authors and reviewers in single-blind peer review. We explored how switching from single-blind to double-blind peer review affected 1) the willingness of experts to review, 2) their publication recommendations, and 3) the quality of review reports. Methods The Finnish Medical Journal switched from single-blind to double-blind peer review in September 2017. The proportion of review invitations that resulted in a received review report was counted. The reviewers’ recommendations of “accept as is”, “minor revision”, “major revision” or “reject” were explored. The content of the reviews was assessed by two experienced reviewers using the Review Quality Instrument modified to apply to both original research and review manuscripts. The study material comprised reviews submitted from September 2017 to February 2018. The controls were the reviews submitted between September 2015 and February 2016 and between September 2016 and February 2017. The reviewers’ recommendations and the scorings of quality assessments were tested with the Chi square test, and the means of quality assessments with the independent-samples t test. Results A total of 118 double-blind first-round reviews of 59 manuscripts were compared with 232 single-blind first-round reviews of 116 manuscripts. The proportion of successful review invitations when reviewing single-blinded was 67%, and when reviewing double-blinded, 66%. When reviewing double-blinded, the reviewers recommended accept as is or minor revision less often than during the control period (59% vs. 73%), and major revision or rejection more often (41% vs 27%, P = 0.010). For the quality assessment, 116 reviews from the double-blind period were compared with 104 reviews conducted between September 2016 and February 2017. On a 1–5 scale (1 poor, 5 excellent), double-blind reviews received higher overall proportion of ratings of 4 and 5 than single-blind reviews (56% vs. 49%, P < 0.001). Means for the overall quality of double-blind reviews were 3.38 (IQR, 3.33–3.44) vs. 3.22 (3.17–3.28; P < 0.001) for single-blind reviews. Conclusions The quality of the reviews conducted double-blind was better than of those conducted single-blind. Switching to double-blind review did not alter the reviewers’ willingness to review. The reviewers became slightly more critical.https://doi.org/10.1186/s41073-023-00140-6Peer reviewQuality of peer reviewSingle-blind peer reviewDouble-blind peer reviewScientific publication |
| spellingShingle | Piitu Parmanne Joonas Laajava Noora Järvinen Terttu Harju Mauri Marttunen Pertti Saloheimo Peer reviewers' willingness to review, their recommendations and quality of reviews after the Finnish Medical Journal switched from single-blind to double-blind peer review Research Integrity and Peer Review Peer review Quality of peer review Single-blind peer review Double-blind peer review Scientific publication |
| title | Peer reviewers' willingness to review, their recommendations and quality of reviews after the Finnish Medical Journal switched from single-blind to double-blind peer review |
| title_full | Peer reviewers' willingness to review, their recommendations and quality of reviews after the Finnish Medical Journal switched from single-blind to double-blind peer review |
| title_fullStr | Peer reviewers' willingness to review, their recommendations and quality of reviews after the Finnish Medical Journal switched from single-blind to double-blind peer review |
| title_full_unstemmed | Peer reviewers' willingness to review, their recommendations and quality of reviews after the Finnish Medical Journal switched from single-blind to double-blind peer review |
| title_short | Peer reviewers' willingness to review, their recommendations and quality of reviews after the Finnish Medical Journal switched from single-blind to double-blind peer review |
| title_sort | peer reviewers willingness to review their recommendations and quality of reviews after the finnish medical journal switched from single blind to double blind peer review |
| topic | Peer review Quality of peer review Single-blind peer review Double-blind peer review Scientific publication |
| url | https://doi.org/10.1186/s41073-023-00140-6 |
| work_keys_str_mv | AT piituparmanne peerreviewerswillingnesstoreviewtheirrecommendationsandqualityofreviewsafterthefinnishmedicaljournalswitchedfromsingleblindtodoubleblindpeerreview AT joonaslaajava peerreviewerswillingnesstoreviewtheirrecommendationsandqualityofreviewsafterthefinnishmedicaljournalswitchedfromsingleblindtodoubleblindpeerreview AT noorajarvinen peerreviewerswillingnesstoreviewtheirrecommendationsandqualityofreviewsafterthefinnishmedicaljournalswitchedfromsingleblindtodoubleblindpeerreview AT terttuharju peerreviewerswillingnesstoreviewtheirrecommendationsandqualityofreviewsafterthefinnishmedicaljournalswitchedfromsingleblindtodoubleblindpeerreview AT maurimarttunen peerreviewerswillingnesstoreviewtheirrecommendationsandqualityofreviewsafterthefinnishmedicaljournalswitchedfromsingleblindtodoubleblindpeerreview AT perttisaloheimo peerreviewerswillingnesstoreviewtheirrecommendationsandqualityofreviewsafterthefinnishmedicaljournalswitchedfromsingleblindtodoubleblindpeerreview |