The Clinical Research Bias Index (CRBI): A novel journal ranking method applied to child health respiratory studies
Abstract Background and Aims Journal impact factor has historically been taken as a proxy for quality. However, this is open to significant manipulation and bias. There is currently not widely adopted, robust journal and paper ranking metric which is focused solely on risk of bias. Methods Risk of b...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Wiley
2022-09-01
|
| Series: | Health Science Reports |
| Subjects: | |
| Online Access: | https://doi.org/10.1002/hsr2.739 |
| _version_ | 1797772357848268800 |
|---|---|
| author | Manishaa Vairavan Andrew Prayle Patrick Davies |
| author_facet | Manishaa Vairavan Andrew Prayle Patrick Davies |
| author_sort | Manishaa Vairavan |
| collection | DOAJ |
| description | Abstract Background and Aims Journal impact factor has historically been taken as a proxy for quality. However, this is open to significant manipulation and bias. There is currently not widely adopted, robust journal and paper ranking metric which is focused solely on risk of bias. Methods Risk of bias data was extracted from all Cochrane database systematic reviews in Child Health, Lungs, and Airways for the years 2017–2019. A novel paper quality score, the Clinical Research Bias Index (CRBI), was applied. Individual paper data were pooled for each journal. A comparison was made to journal impact factors, individual paper citations, reads, and altmetric scores. Results 927 papers were analyzed for risk of bias. 119 (12·8%) scored a CRBI of 100%, with a mean score of 70%. A journal's overall CRBI risk of bias score was poorly correlated with impact factor (r 0.25). Citations (r 0.02), and reads (r 0.01) of individual papers showed very little association with the paper's risk of bias. Likewise, reads were not correlated with citations (r 0.03). H‐index and Altmetric scores were similarly poorly correlated with CRBI. Conclusion The novel research quality tool CRBI demonstrates the poor correlation between journal impact factor, citations, and risk of bias. Journal and paper ranking metrics should ensure that they are fit for purpose, and enable the dissemination of high‐quality research for the benefit of patients. We propose the CRBI as a potential solution which is resistant to manipulation and will reward the creation and publication of bias‐free research. |
| first_indexed | 2024-03-12T21:50:44Z |
| format | Article |
| id | doaj.art-f638bd34ca4842078642d88b4bcbcfe3 |
| institution | Directory Open Access Journal |
| issn | 2398-8835 |
| language | English |
| last_indexed | 2024-03-12T21:50:44Z |
| publishDate | 2022-09-01 |
| publisher | Wiley |
| record_format | Article |
| series | Health Science Reports |
| spelling | doaj.art-f638bd34ca4842078642d88b4bcbcfe32023-07-26T04:11:54ZengWileyHealth Science Reports2398-88352022-09-0155n/an/a10.1002/hsr2.739The Clinical Research Bias Index (CRBI): A novel journal ranking method applied to child health respiratory studiesManishaa Vairavan0Andrew Prayle1Patrick Davies2University of Nottingham School of Medicine Nottingham UKUniversity of Nottingham School of Medicine Nottingham UKUniversity of Nottingham School of Medicine Nottingham UKAbstract Background and Aims Journal impact factor has historically been taken as a proxy for quality. However, this is open to significant manipulation and bias. There is currently not widely adopted, robust journal and paper ranking metric which is focused solely on risk of bias. Methods Risk of bias data was extracted from all Cochrane database systematic reviews in Child Health, Lungs, and Airways for the years 2017–2019. A novel paper quality score, the Clinical Research Bias Index (CRBI), was applied. Individual paper data were pooled for each journal. A comparison was made to journal impact factors, individual paper citations, reads, and altmetric scores. Results 927 papers were analyzed for risk of bias. 119 (12·8%) scored a CRBI of 100%, with a mean score of 70%. A journal's overall CRBI risk of bias score was poorly correlated with impact factor (r 0.25). Citations (r 0.02), and reads (r 0.01) of individual papers showed very little association with the paper's risk of bias. Likewise, reads were not correlated with citations (r 0.03). H‐index and Altmetric scores were similarly poorly correlated with CRBI. Conclusion The novel research quality tool CRBI demonstrates the poor correlation between journal impact factor, citations, and risk of bias. Journal and paper ranking metrics should ensure that they are fit for purpose, and enable the dissemination of high‐quality research for the benefit of patients. We propose the CRBI as a potential solution which is resistant to manipulation and will reward the creation and publication of bias‐free research.https://doi.org/10.1002/hsr2.739biascitationsImpact Factorquality |
| spellingShingle | Manishaa Vairavan Andrew Prayle Patrick Davies The Clinical Research Bias Index (CRBI): A novel journal ranking method applied to child health respiratory studies Health Science Reports bias citations Impact Factor quality |
| title | The Clinical Research Bias Index (CRBI): A novel journal ranking method applied to child health respiratory studies |
| title_full | The Clinical Research Bias Index (CRBI): A novel journal ranking method applied to child health respiratory studies |
| title_fullStr | The Clinical Research Bias Index (CRBI): A novel journal ranking method applied to child health respiratory studies |
| title_full_unstemmed | The Clinical Research Bias Index (CRBI): A novel journal ranking method applied to child health respiratory studies |
| title_short | The Clinical Research Bias Index (CRBI): A novel journal ranking method applied to child health respiratory studies |
| title_sort | clinical research bias index crbi a novel journal ranking method applied to child health respiratory studies |
| topic | bias citations Impact Factor quality |
| url | https://doi.org/10.1002/hsr2.739 |
| work_keys_str_mv | AT manishaavairavan theclinicalresearchbiasindexcrbianoveljournalrankingmethodappliedtochildhealthrespiratorystudies AT andrewprayle theclinicalresearchbiasindexcrbianoveljournalrankingmethodappliedtochildhealthrespiratorystudies AT patrickdavies theclinicalresearchbiasindexcrbianoveljournalrankingmethodappliedtochildhealthrespiratorystudies AT manishaavairavan clinicalresearchbiasindexcrbianoveljournalrankingmethodappliedtochildhealthrespiratorystudies AT andrewprayle clinicalresearchbiasindexcrbianoveljournalrankingmethodappliedtochildhealthrespiratorystudies AT patrickdavies clinicalresearchbiasindexcrbianoveljournalrankingmethodappliedtochildhealthrespiratorystudies |