Inter-rater agreement and reliability of the COSMIN (COnsensus-based Standards for the selection of health status Measurement Instruments) Checklist
<p style="text-align:justify;"> <b>Background:</b> The COSMIN checklist is a tool for evaluating the methodological quality of studies on measurement properties of health-related patient-reported outcomes. The aim of this study is to determine the inter-rater agreement a...
| Автори: | , , , , , , , , |
|---|---|
| Формат: | Journal article |
| Мова: | English |
| Опубліковано: |
BioMed Central
2010
|
| _version_ | 1826265299478904832 |
|---|---|
| author | Mokkink, L Terwee, C Gibbons, E Stratford, P Alonso, J Patrick, D Knol, D Bouter, L De Vet, H |
| author_facet | Mokkink, L Terwee, C Gibbons, E Stratford, P Alonso, J Patrick, D Knol, D Bouter, L De Vet, H |
| author_sort | Mokkink, L |
| collection | OXFORD |
| description | <p style="text-align:justify;"> <b>Background:</b> The COSMIN checklist is a tool for evaluating the methodological quality of studies on measurement properties of health-related patient-reported outcomes. The aim of this study is to determine the inter-rater agreement and reliability of each item score of the COSMIN checklist (n = 114).<br/><br/> <b>Methods:</b> 75 articles evaluating measurement properties were randomly selected from the bibliographic database compiled by the Patient-Reported Outcome Measurement Group, Oxford, UK. Raters were asked to assess the methodological quality of three articles, using the COSMIN checklist. In a one-way design, percentage agreement and intraclass kappa coefficients or quadratic-weighted kappa coefficients were calculated for each item.<br/><br/> <b>Results:</b> 88 raters participated. Of the 75 selected articles, 26 articles were rated by four to six participants, and 49 by two or three participants. Overall, percentage agreement was appropriate (68% was above 80% agreement), and the kappa coefficients for the COSMIN items were low (61% was below 0.40, 6% was above 0.75). Reasons for low inter-rater agreement were need for subjective judgement, and accustom to different standards, terminology and definitions.<br/><br/> <b>Conclusions:</b> Results indicated that raters often choose the same response option, but that it is difficult on item level to distinguish between articles. When using the COSMIN checklist in a systematic review, we recommend getting some training and experience, completing it by two independent raters, and reaching consensus on one final rating. Instructions for using the checklist are improved. </p> |
| first_indexed | 2024-03-06T20:21:30Z |
| format | Journal article |
| id | oxford-uuid:2def9d2f-a841-4e12-a6fb-8f1593c11bc3 |
| institution | University of Oxford |
| language | English |
| last_indexed | 2024-03-06T20:21:30Z |
| publishDate | 2010 |
| publisher | BioMed Central |
| record_format | dspace |
| spelling | oxford-uuid:2def9d2f-a841-4e12-a6fb-8f1593c11bc32022-03-26T12:46:07ZInter-rater agreement and reliability of the COSMIN (COnsensus-based Standards for the selection of health status Measurement Instruments) ChecklistJournal articlehttp://purl.org/coar/resource_type/c_dcae04bcuuid:2def9d2f-a841-4e12-a6fb-8f1593c11bc3EnglishSymplectic Elements at OxfordBioMed Central2010Mokkink, LTerwee, CGibbons, EStratford, PAlonso, JPatrick, DKnol, DBouter, LDe Vet, H <p style="text-align:justify;"> <b>Background:</b> The COSMIN checklist is a tool for evaluating the methodological quality of studies on measurement properties of health-related patient-reported outcomes. The aim of this study is to determine the inter-rater agreement and reliability of each item score of the COSMIN checklist (n = 114).<br/><br/> <b>Methods:</b> 75 articles evaluating measurement properties were randomly selected from the bibliographic database compiled by the Patient-Reported Outcome Measurement Group, Oxford, UK. Raters were asked to assess the methodological quality of three articles, using the COSMIN checklist. In a one-way design, percentage agreement and intraclass kappa coefficients or quadratic-weighted kappa coefficients were calculated for each item.<br/><br/> <b>Results:</b> 88 raters participated. Of the 75 selected articles, 26 articles were rated by four to six participants, and 49 by two or three participants. Overall, percentage agreement was appropriate (68% was above 80% agreement), and the kappa coefficients for the COSMIN items were low (61% was below 0.40, 6% was above 0.75). Reasons for low inter-rater agreement were need for subjective judgement, and accustom to different standards, terminology and definitions.<br/><br/> <b>Conclusions:</b> Results indicated that raters often choose the same response option, but that it is difficult on item level to distinguish between articles. When using the COSMIN checklist in a systematic review, we recommend getting some training and experience, completing it by two independent raters, and reaching consensus on one final rating. Instructions for using the checklist are improved. </p> |
| spellingShingle | Mokkink, L Terwee, C Gibbons, E Stratford, P Alonso, J Patrick, D Knol, D Bouter, L De Vet, H Inter-rater agreement and reliability of the COSMIN (COnsensus-based Standards for the selection of health status Measurement Instruments) Checklist |
| title | Inter-rater agreement and reliability of the COSMIN (COnsensus-based Standards for the selection of health status Measurement Instruments) Checklist |
| title_full | Inter-rater agreement and reliability of the COSMIN (COnsensus-based Standards for the selection of health status Measurement Instruments) Checklist |
| title_fullStr | Inter-rater agreement and reliability of the COSMIN (COnsensus-based Standards for the selection of health status Measurement Instruments) Checklist |
| title_full_unstemmed | Inter-rater agreement and reliability of the COSMIN (COnsensus-based Standards for the selection of health status Measurement Instruments) Checklist |
| title_short | Inter-rater agreement and reliability of the COSMIN (COnsensus-based Standards for the selection of health status Measurement Instruments) Checklist |
| title_sort | inter rater agreement and reliability of the cosmin consensus based standards for the selection of health status measurement instruments checklist |
| work_keys_str_mv | AT mokkinkl interrateragreementandreliabilityofthecosminconsensusbasedstandardsfortheselectionofhealthstatusmeasurementinstrumentschecklist AT terweec interrateragreementandreliabilityofthecosminconsensusbasedstandardsfortheselectionofhealthstatusmeasurementinstrumentschecklist AT gibbonse interrateragreementandreliabilityofthecosminconsensusbasedstandardsfortheselectionofhealthstatusmeasurementinstrumentschecklist AT stratfordp interrateragreementandreliabilityofthecosminconsensusbasedstandardsfortheselectionofhealthstatusmeasurementinstrumentschecklist AT alonsoj interrateragreementandreliabilityofthecosminconsensusbasedstandardsfortheselectionofhealthstatusmeasurementinstrumentschecklist AT patrickd interrateragreementandreliabilityofthecosminconsensusbasedstandardsfortheselectionofhealthstatusmeasurementinstrumentschecklist AT knold interrateragreementandreliabilityofthecosminconsensusbasedstandardsfortheselectionofhealthstatusmeasurementinstrumentschecklist AT bouterl interrateragreementandreliabilityofthecosminconsensusbasedstandardsfortheselectionofhealthstatusmeasurementinstrumentschecklist AT deveth interrateragreementandreliabilityofthecosminconsensusbasedstandardsfortheselectionofhealthstatusmeasurementinstrumentschecklist |