| 要約: | How trustworthy are peer reviews by applicant-nominated assessors (ANAs)? For Australian Research Council (ARC) proposals (N = 2,330) with at least one ANA and one assessor nominated by the funding panel (PNAs), ANAs gave substantially higher ratings in all nine discipline panels (covering sciences, social sciences, and humanities). Compared to reviews by PNAs, ANA ratings were less related to ratings by other assessors, less related to the ARC final assessment, and contributed to the unreliability of peer reviews. Furthermore, when the same assessor was both an ANA and PNA for different proposals, ratings in the role of ANA were biased whereas those by the same person in the role of PNA were not. ANA ratings of ARC grant proposals are biased, inflated, unreliable, and invalid, leading the ARC to abandon use of ANAs. Particularly if replicated in other situations, the results have important implications for other evaluations based on ANAs. © The Australian Psychological Society Ltd.
|
|---|