Calculating the probability of random sampling for continuous variables in submitted or published randomised controlled trials
In a previous paper, one of the authors (JBC) used a chi-squared method to analyse the means (SD) of baseline variables, such as height or weight, from randomised controlled trials by Fujii et al., concluding that the probabilities that the reported distributions arose by chance were infinitesimally...
| Main Authors: | , , , , |
|---|---|
| Format: | Journal article |
| Language: | English |
| Published: |
Wiley
2015
|
| _version_ | 1797050206149148672 |
|---|---|
| author | Carlisle, J Dexter, F Pandit, J Shafer, S Yentis, S |
| author_facet | Carlisle, J Dexter, F Pandit, J Shafer, S Yentis, S |
| author_sort | Carlisle, J |
| collection | OXFORD |
| description | In a previous paper, one of the authors (JBC) used a chi-squared method to analyse the means (SD) of baseline variables, such as height or weight, from randomised controlled trials by Fujii et al., concluding that the probabilities that the reported distributions arose by chance were infinitesimally small. Subsequent testing of that chi-squared method, using simulation, suggested that the method was incorrect. This paper corrects the chi-squared method and tests its performance and the performance of Monte Carlo simulations and ANOVA to analyse the probability of random sampling. The corrected chi-squared method and ANOVA method became inaccurate when applied to means that were reported imprecisely. Monte Carlo simulations confirmed that baseline data from 158 randomised controlled trials by Fujii et al. were different to those from 329 trials published by other authors and that the distribution of Fujii et al.'s data were different to the expected distribution, both p < 10(-16) . The number of Fujii randomised controlled trials with unlikely distributions was less with Monte Carlo simulation than with the 2012 chi-squared method: 102 vs 117 trials with p < 0.05; 60 vs 86 for p < 0.01; 30 vs 56 for p < 0.001; and 12 vs 24 for p < 0.00001, respectively. The Monte Carlo analysis nevertheless confirmed the original conclusion that the distribution of the data presented by Fujii et al. was extremely unlikely to have arisen from observed data. The Monte Carlo analysis may be an appropriate screening tool to check for non-random (i.e. unreliable) data in randomised controlled trials submitted to journals. |
| first_indexed | 2024-03-06T18:01:44Z |
| format | Journal article |
| id | oxford-uuid:000401ff-07b4-4be2-afc1-ccd4b2684655 |
| institution | University of Oxford |
| language | English |
| last_indexed | 2024-03-06T18:01:44Z |
| publishDate | 2015 |
| publisher | Wiley |
| record_format | dspace |
| spelling | oxford-uuid:000401ff-07b4-4be2-afc1-ccd4b26846552022-03-26T08:27:10ZCalculating the probability of random sampling for continuous variables in submitted or published randomised controlled trialsJournal articlehttp://purl.org/coar/resource_type/c_dcae04bcuuid:000401ff-07b4-4be2-afc1-ccd4b2684655EnglishSymplectic Elements at OxfordWiley2015Carlisle, JDexter, FPandit, JShafer, SYentis, SIn a previous paper, one of the authors (JBC) used a chi-squared method to analyse the means (SD) of baseline variables, such as height or weight, from randomised controlled trials by Fujii et al., concluding that the probabilities that the reported distributions arose by chance were infinitesimally small. Subsequent testing of that chi-squared method, using simulation, suggested that the method was incorrect. This paper corrects the chi-squared method and tests its performance and the performance of Monte Carlo simulations and ANOVA to analyse the probability of random sampling. The corrected chi-squared method and ANOVA method became inaccurate when applied to means that were reported imprecisely. Monte Carlo simulations confirmed that baseline data from 158 randomised controlled trials by Fujii et al. were different to those from 329 trials published by other authors and that the distribution of Fujii et al.'s data were different to the expected distribution, both p < 10(-16) . The number of Fujii randomised controlled trials with unlikely distributions was less with Monte Carlo simulation than with the 2012 chi-squared method: 102 vs 117 trials with p < 0.05; 60 vs 86 for p < 0.01; 30 vs 56 for p < 0.001; and 12 vs 24 for p < 0.00001, respectively. The Monte Carlo analysis nevertheless confirmed the original conclusion that the distribution of the data presented by Fujii et al. was extremely unlikely to have arisen from observed data. The Monte Carlo analysis may be an appropriate screening tool to check for non-random (i.e. unreliable) data in randomised controlled trials submitted to journals. |
| spellingShingle | Carlisle, J Dexter, F Pandit, J Shafer, S Yentis, S Calculating the probability of random sampling for continuous variables in submitted or published randomised controlled trials |
| title | Calculating the probability of random sampling for continuous variables in submitted or published randomised controlled trials |
| title_full | Calculating the probability of random sampling for continuous variables in submitted or published randomised controlled trials |
| title_fullStr | Calculating the probability of random sampling for continuous variables in submitted or published randomised controlled trials |
| title_full_unstemmed | Calculating the probability of random sampling for continuous variables in submitted or published randomised controlled trials |
| title_short | Calculating the probability of random sampling for continuous variables in submitted or published randomised controlled trials |
| title_sort | calculating the probability of random sampling for continuous variables in submitted or published randomised controlled trials |
| work_keys_str_mv | AT carlislej calculatingtheprobabilityofrandomsamplingforcontinuousvariablesinsubmittedorpublishedrandomisedcontrolledtrials AT dexterf calculatingtheprobabilityofrandomsamplingforcontinuousvariablesinsubmittedorpublishedrandomisedcontrolledtrials AT panditj calculatingtheprobabilityofrandomsamplingforcontinuousvariablesinsubmittedorpublishedrandomisedcontrolledtrials AT shafers calculatingtheprobabilityofrandomsamplingforcontinuousvariablesinsubmittedorpublishedrandomisedcontrolledtrials AT yentiss calculatingtheprobabilityofrandomsamplingforcontinuousvariablesinsubmittedorpublishedrandomisedcontrolledtrials |