A fairer way to compare researchers at any career stage and in any discipline using open-access citation data.
The pursuit of simple, yet fair, unbiased, and objective measures of researcher performance has occupied bibliometricians and the research community as a whole for decades. However, despite the diversity of available metrics, most are either complex to calculate or not readily applied in the most co...
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Public Library of Science (PLoS)
2021-01-01
|
| Series: | PLoS ONE |
| Online Access: | https://doi.org/10.1371/journal.pone.0257141 |
| _version_ | 1818436245299134464 |
|---|---|
| author | Corey J A Bradshaw Justin M Chalker Stefani A Crabtree Bart A Eijkelkamp John A Long Justine R Smith Kate Trinajstic Vera Weisbecker |
| author_facet | Corey J A Bradshaw Justin M Chalker Stefani A Crabtree Bart A Eijkelkamp John A Long Justine R Smith Kate Trinajstic Vera Weisbecker |
| author_sort | Corey J A Bradshaw |
| collection | DOAJ |
| description | The pursuit of simple, yet fair, unbiased, and objective measures of researcher performance has occupied bibliometricians and the research community as a whole for decades. However, despite the diversity of available metrics, most are either complex to calculate or not readily applied in the most common assessment exercises (e.g., grant assessment, job applications). The ubiquity of metrics like the h-index (h papers with at least h citations) and its time-corrected variant, the m-quotient (h-index ÷ number of years publishing) therefore reflect the ease of use rather than their capacity to differentiate researchers fairly among disciplines, career stage, or gender. We address this problem here by defining an easily calculated index based on publicly available citation data (Google Scholar) that corrects for most biases and allows assessors to compare researchers at any stage of their career and from any discipline on the same scale. Our ε'-index violates fewer statistical assumptions relative to other metrics when comparing groups of researchers, and can be easily modified to remove inherent gender biases in citation data. We demonstrate the utility of the ε'-index using a sample of 480 researchers with Google Scholar profiles, stratified evenly into eight disciplines (archaeology, chemistry, ecology, evolution and development, geology, microbiology, ophthalmology, palaeontology), three career stages (early, mid-, late-career), and two genders. We advocate the use of the ε'-index whenever assessors must compare research performance among researchers of different backgrounds, but emphasize that no single index should be used exclusively to rank researcher capability. |
| first_indexed | 2024-12-14T17:05:43Z |
| format | Article |
| id | doaj.art-f96947b3209f4924bf06a2c21eee0f07 |
| institution | Directory Open Access Journal |
| issn | 1932-6203 |
| language | English |
| last_indexed | 2024-12-14T17:05:43Z |
| publishDate | 2021-01-01 |
| publisher | Public Library of Science (PLoS) |
| record_format | Article |
| series | PLoS ONE |
| spelling | doaj.art-f96947b3209f4924bf06a2c21eee0f072022-12-21T22:53:44ZengPublic Library of Science (PLoS)PLoS ONE1932-62032021-01-01169e025714110.1371/journal.pone.0257141A fairer way to compare researchers at any career stage and in any discipline using open-access citation data.Corey J A BradshawJustin M ChalkerStefani A CrabtreeBart A EijkelkampJohn A LongJustine R SmithKate TrinajsticVera WeisbeckerThe pursuit of simple, yet fair, unbiased, and objective measures of researcher performance has occupied bibliometricians and the research community as a whole for decades. However, despite the diversity of available metrics, most are either complex to calculate or not readily applied in the most common assessment exercises (e.g., grant assessment, job applications). The ubiquity of metrics like the h-index (h papers with at least h citations) and its time-corrected variant, the m-quotient (h-index ÷ number of years publishing) therefore reflect the ease of use rather than their capacity to differentiate researchers fairly among disciplines, career stage, or gender. We address this problem here by defining an easily calculated index based on publicly available citation data (Google Scholar) that corrects for most biases and allows assessors to compare researchers at any stage of their career and from any discipline on the same scale. Our ε'-index violates fewer statistical assumptions relative to other metrics when comparing groups of researchers, and can be easily modified to remove inherent gender biases in citation data. We demonstrate the utility of the ε'-index using a sample of 480 researchers with Google Scholar profiles, stratified evenly into eight disciplines (archaeology, chemistry, ecology, evolution and development, geology, microbiology, ophthalmology, palaeontology), three career stages (early, mid-, late-career), and two genders. We advocate the use of the ε'-index whenever assessors must compare research performance among researchers of different backgrounds, but emphasize that no single index should be used exclusively to rank researcher capability.https://doi.org/10.1371/journal.pone.0257141 |
| spellingShingle | Corey J A Bradshaw Justin M Chalker Stefani A Crabtree Bart A Eijkelkamp John A Long Justine R Smith Kate Trinajstic Vera Weisbecker A fairer way to compare researchers at any career stage and in any discipline using open-access citation data. PLoS ONE |
| title | A fairer way to compare researchers at any career stage and in any discipline using open-access citation data. |
| title_full | A fairer way to compare researchers at any career stage and in any discipline using open-access citation data. |
| title_fullStr | A fairer way to compare researchers at any career stage and in any discipline using open-access citation data. |
| title_full_unstemmed | A fairer way to compare researchers at any career stage and in any discipline using open-access citation data. |
| title_short | A fairer way to compare researchers at any career stage and in any discipline using open-access citation data. |
| title_sort | fairer way to compare researchers at any career stage and in any discipline using open access citation data |
| url | https://doi.org/10.1371/journal.pone.0257141 |
| work_keys_str_mv | AT coreyjabradshaw afairerwaytocompareresearchersatanycareerstageandinanydisciplineusingopenaccesscitationdata AT justinmchalker afairerwaytocompareresearchersatanycareerstageandinanydisciplineusingopenaccesscitationdata AT stefaniacrabtree afairerwaytocompareresearchersatanycareerstageandinanydisciplineusingopenaccesscitationdata AT bartaeijkelkamp afairerwaytocompareresearchersatanycareerstageandinanydisciplineusingopenaccesscitationdata AT johnalong afairerwaytocompareresearchersatanycareerstageandinanydisciplineusingopenaccesscitationdata AT justinersmith afairerwaytocompareresearchersatanycareerstageandinanydisciplineusingopenaccesscitationdata AT katetrinajstic afairerwaytocompareresearchersatanycareerstageandinanydisciplineusingopenaccesscitationdata AT veraweisbecker afairerwaytocompareresearchersatanycareerstageandinanydisciplineusingopenaccesscitationdata AT coreyjabradshaw fairerwaytocompareresearchersatanycareerstageandinanydisciplineusingopenaccesscitationdata AT justinmchalker fairerwaytocompareresearchersatanycareerstageandinanydisciplineusingopenaccesscitationdata AT stefaniacrabtree fairerwaytocompareresearchersatanycareerstageandinanydisciplineusingopenaccesscitationdata AT bartaeijkelkamp fairerwaytocompareresearchersatanycareerstageandinanydisciplineusingopenaccesscitationdata AT johnalong fairerwaytocompareresearchersatanycareerstageandinanydisciplineusingopenaccesscitationdata AT justinersmith fairerwaytocompareresearchersatanycareerstageandinanydisciplineusingopenaccesscitationdata AT katetrinajstic fairerwaytocompareresearchersatanycareerstageandinanydisciplineusingopenaccesscitationdata AT veraweisbecker fairerwaytocompareresearchersatanycareerstageandinanydisciplineusingopenaccesscitationdata |