No relationship between researcher impact and replication effect: an analysis of five studies with 100 replications
What explanation is there when teams of researchers are unable to successfully replicate already established ‘canonical’ findings? One suggestion that has been put forward, but left largely untested, is that those researchers who fail to replicate prior studies are of low ‘expertise and diligence’ a...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
PeerJ Inc.
2020-03-01
|
| Series: | PeerJ |
| Subjects: | |
| Online Access: | https://peerj.com/articles/8014.pdf |
| _version_ | 1797422181839273984 |
|---|---|
| author | John Protzko Jonathan W. Schooler |
| author_facet | John Protzko Jonathan W. Schooler |
| author_sort | John Protzko |
| collection | DOAJ |
| description | What explanation is there when teams of researchers are unable to successfully replicate already established ‘canonical’ findings? One suggestion that has been put forward, but left largely untested, is that those researchers who fail to replicate prior studies are of low ‘expertise and diligence’ and lack the skill necessary to successfully replicate the conditions of the original experiment. Here we examine the replication success of 100 scientists of differing ‘expertise and diligence’ who attempted to replicate five different studies. Using a bibliometric tool (h-index) as our indicator of researcher ‘expertise and diligence’, we examine whether this was predictive of replication success. Although there was substantial variability in replication success and in the h-factor of the investigators, we find no relationship between these variables. The present results provide no evidence for the hypothesis that systematic replications fail because of low ‘expertise and diligence’ among replicators. |
| first_indexed | 2024-03-09T07:28:29Z |
| format | Article |
| id | doaj.art-ac1edf3ee2064beaaf0f561f4ffab634 |
| institution | Directory Open Access Journal |
| issn | 2167-8359 |
| language | English |
| last_indexed | 2024-03-09T07:28:29Z |
| publishDate | 2020-03-01 |
| publisher | PeerJ Inc. |
| record_format | Article |
| series | PeerJ |
| spelling | doaj.art-ac1edf3ee2064beaaf0f561f4ffab6342023-12-03T06:47:20ZengPeerJ Inc.PeerJ2167-83592020-03-018e801410.7717/peerj.8014No relationship between researcher impact and replication effect: an analysis of five studies with 100 replicationsJohn Protzko0Jonathan W. Schooler1Psychological and Brain Sciences, University of California, Santa Barbara, Santa Barbara, CA, United States of AmericaPsychological and Brain Sciences, University of California, Santa Barbara, Santa Barbara, CA, United States of AmericaWhat explanation is there when teams of researchers are unable to successfully replicate already established ‘canonical’ findings? One suggestion that has been put forward, but left largely untested, is that those researchers who fail to replicate prior studies are of low ‘expertise and diligence’ and lack the skill necessary to successfully replicate the conditions of the original experiment. Here we examine the replication success of 100 scientists of differing ‘expertise and diligence’ who attempted to replicate five different studies. Using a bibliometric tool (h-index) as our indicator of researcher ‘expertise and diligence’, we examine whether this was predictive of replication success. Although there was substantial variability in replication success and in the h-factor of the investigators, we find no relationship between these variables. The present results provide no evidence for the hypothesis that systematic replications fail because of low ‘expertise and diligence’ among replicators.https://peerj.com/articles/8014.pdfReproducibilityBibliometricsMetascienceh-indexRegistered Replication ReportsScientometrics |
| spellingShingle | John Protzko Jonathan W. Schooler No relationship between researcher impact and replication effect: an analysis of five studies with 100 replications PeerJ Reproducibility Bibliometrics Metascience h-index Registered Replication Reports Scientometrics |
| title | No relationship between researcher impact and replication effect: an analysis of five studies with 100 replications |
| title_full | No relationship between researcher impact and replication effect: an analysis of five studies with 100 replications |
| title_fullStr | No relationship between researcher impact and replication effect: an analysis of five studies with 100 replications |
| title_full_unstemmed | No relationship between researcher impact and replication effect: an analysis of five studies with 100 replications |
| title_short | No relationship between researcher impact and replication effect: an analysis of five studies with 100 replications |
| title_sort | no relationship between researcher impact and replication effect an analysis of five studies with 100 replications |
| topic | Reproducibility Bibliometrics Metascience h-index Registered Replication Reports Scientometrics |
| url | https://peerj.com/articles/8014.pdf |
| work_keys_str_mv | AT johnprotzko norelationshipbetweenresearcherimpactandreplicationeffectananalysisoffivestudieswith100replications AT jonathanwschooler norelationshipbetweenresearcherimpactandreplicationeffectananalysisoffivestudieswith100replications |