A descriptive study of samples sizes used in agreement studies published in the PubMed repository
Abstract Introduction A sample size justification is required for all studies and should give the minimum number of subjects to be recruited for the study to achieve its primary objective. The aim of this review is to describe sample sizes from agreement studies with continuous or categorical endpoi...
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
BMC
2022-09-01
|
| Series: | BMC Medical Research Methodology |
| Subjects: | |
| Online Access: | https://doi.org/10.1186/s12874-022-01723-5 |
| _version_ | 1811208687208890368 |
|---|---|
| author | Oscar Han Hao Wei Tan Steven Julious Laura Sutton Richard Jacques Ellen Lee Jen Lewis Stephen Walters |
| author_facet | Oscar Han Hao Wei Tan Steven Julious Laura Sutton Richard Jacques Ellen Lee Jen Lewis Stephen Walters |
| author_sort | Oscar Han |
| collection | DOAJ |
| description | Abstract Introduction A sample size justification is required for all studies and should give the minimum number of subjects to be recruited for the study to achieve its primary objective. The aim of this review is to describe sample sizes from agreement studies with continuous or categorical endpoints and different methods of assessing agreement, and to determine whether sample size justification was provided. Methods Data were gathered from the PubMed repository with a time interval of 28th September 2018 to 28th September 2020. The search returned 5257 studies of which 82 studies were eligible for final assessment after duplicates and ineligible studies were excluded. Results We observed a wide range of sample sizes. Forty-six studies (56%) used a continuous outcome measure, 28 (34%) used categorical and eight (10%) used both. Median sample sizes were 50 (IQR 25 to 100) for continuous endpoints and 119 (IQR 50 to 271) for categorical endpoints. Bland–Altman limits of agreement (median sample size 65; IQR 35 to 124) were the most common method of statistical analysis for continuous variables and Kappa coefficients for categorical variables (median sample size 71; IQR 50 to 233). Of the 82 studies assessed, only 27 (33%) gave justification for their sample size. Conclusions Despite the importance of a sample size justification, we found that two-thirds of agreement studies did not provide one. We recommend that all agreement studies provide rationale for their sample size even if they do not include a formal sample size calculation. |
| first_indexed | 2024-04-12T04:26:51Z |
| format | Article |
| id | doaj.art-8736a881484d4dc6b8148d999adc99f1 |
| institution | Directory Open Access Journal |
| issn | 1471-2288 |
| language | English |
| last_indexed | 2024-04-12T04:26:51Z |
| publishDate | 2022-09-01 |
| publisher | BMC |
| record_format | Article |
| series | BMC Medical Research Methodology |
| spelling | doaj.art-8736a881484d4dc6b8148d999adc99f12022-12-22T03:48:04ZengBMCBMC Medical Research Methodology1471-22882022-09-012211610.1186/s12874-022-01723-5A descriptive study of samples sizes used in agreement studies published in the PubMed repositoryOscar Han0Hao Wei Tan1Steven Julious2Laura Sutton3Richard Jacques4Ellen Lee5Jen Lewis6Stephen Walters7School of Health and Related Research, University of SheffieldSchool of Health and Related Research, University of SheffieldSchool of Health and Related Research, University of SheffieldSchool of Health and Related Research, University of SheffieldSchool of Health and Related Research, University of SheffieldSchool of Health and Related Research, University of SheffieldSchool of Health and Related Research, University of SheffieldSchool of Health and Related Research, University of SheffieldAbstract Introduction A sample size justification is required for all studies and should give the minimum number of subjects to be recruited for the study to achieve its primary objective. The aim of this review is to describe sample sizes from agreement studies with continuous or categorical endpoints and different methods of assessing agreement, and to determine whether sample size justification was provided. Methods Data were gathered from the PubMed repository with a time interval of 28th September 2018 to 28th September 2020. The search returned 5257 studies of which 82 studies were eligible for final assessment after duplicates and ineligible studies were excluded. Results We observed a wide range of sample sizes. Forty-six studies (56%) used a continuous outcome measure, 28 (34%) used categorical and eight (10%) used both. Median sample sizes were 50 (IQR 25 to 100) for continuous endpoints and 119 (IQR 50 to 271) for categorical endpoints. Bland–Altman limits of agreement (median sample size 65; IQR 35 to 124) were the most common method of statistical analysis for continuous variables and Kappa coefficients for categorical variables (median sample size 71; IQR 50 to 233). Of the 82 studies assessed, only 27 (33%) gave justification for their sample size. Conclusions Despite the importance of a sample size justification, we found that two-thirds of agreement studies did not provide one. We recommend that all agreement studies provide rationale for their sample size even if they do not include a formal sample size calculation.https://doi.org/10.1186/s12874-022-01723-5AgreementDesignMethod comparisonSample sizeTest–retest |
| spellingShingle | Oscar Han Hao Wei Tan Steven Julious Laura Sutton Richard Jacques Ellen Lee Jen Lewis Stephen Walters A descriptive study of samples sizes used in agreement studies published in the PubMed repository BMC Medical Research Methodology Agreement Design Method comparison Sample size Test–retest |
| title | A descriptive study of samples sizes used in agreement studies published in the PubMed repository |
| title_full | A descriptive study of samples sizes used in agreement studies published in the PubMed repository |
| title_fullStr | A descriptive study of samples sizes used in agreement studies published in the PubMed repository |
| title_full_unstemmed | A descriptive study of samples sizes used in agreement studies published in the PubMed repository |
| title_short | A descriptive study of samples sizes used in agreement studies published in the PubMed repository |
| title_sort | descriptive study of samples sizes used in agreement studies published in the pubmed repository |
| topic | Agreement Design Method comparison Sample size Test–retest |
| url | https://doi.org/10.1186/s12874-022-01723-5 |
| work_keys_str_mv | AT oscarhan adescriptivestudyofsamplessizesusedinagreementstudiespublishedinthepubmedrepository AT haoweitan adescriptivestudyofsamplessizesusedinagreementstudiespublishedinthepubmedrepository AT stevenjulious adescriptivestudyofsamplessizesusedinagreementstudiespublishedinthepubmedrepository AT laurasutton adescriptivestudyofsamplessizesusedinagreementstudiespublishedinthepubmedrepository AT richardjacques adescriptivestudyofsamplessizesusedinagreementstudiespublishedinthepubmedrepository AT ellenlee adescriptivestudyofsamplessizesusedinagreementstudiespublishedinthepubmedrepository AT jenlewis adescriptivestudyofsamplessizesusedinagreementstudiespublishedinthepubmedrepository AT stephenwalters adescriptivestudyofsamplessizesusedinagreementstudiespublishedinthepubmedrepository AT oscarhan descriptivestudyofsamplessizesusedinagreementstudiespublishedinthepubmedrepository AT haoweitan descriptivestudyofsamplessizesusedinagreementstudiespublishedinthepubmedrepository AT stevenjulious descriptivestudyofsamplessizesusedinagreementstudiespublishedinthepubmedrepository AT laurasutton descriptivestudyofsamplessizesusedinagreementstudiespublishedinthepubmedrepository AT richardjacques descriptivestudyofsamplessizesusedinagreementstudiespublishedinthepubmedrepository AT ellenlee descriptivestudyofsamplessizesusedinagreementstudiespublishedinthepubmedrepository AT jenlewis descriptivestudyofsamplessizesusedinagreementstudiespublishedinthepubmedrepository AT stephenwalters descriptivestudyofsamplessizesusedinagreementstudiespublishedinthepubmedrepository |