Evaluation of a Meta-Analysis of Ambient Air Quality as a Risk Factor for Asthma Exacerbation
Background: An irreproducibility crisis currently afflicts a wide range of scientific disciplines, including public health and biomedical science. A study was undertaken to assess the reliability of a meta-analysis examining whether air quality components (carbon monoxide, particulate matter 10 µm a...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2021-06-01
|
| Series: | Journal of Respiration |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2673-527X/1/3/17 |
| _version_ | 1797518611060883456 |
|---|---|
| author | Warren Kindzierski Stanley Young Terry Meyer John Dunn |
| author_facet | Warren Kindzierski Stanley Young Terry Meyer John Dunn |
| author_sort | Warren Kindzierski |
| collection | DOAJ |
| description | Background: An irreproducibility crisis currently afflicts a wide range of scientific disciplines, including public health and biomedical science. A study was undertaken to assess the reliability of a meta-analysis examining whether air quality components (carbon monoxide, particulate matter 10 µm and 2.5 µm (PM10 and PM2.5), sulfur dioxide, nitrogen dioxide and ozone) are risk factors for asthma exacerbation. Methods: The number of statistical tests and models were counted in 17 randomly selected base papers from 87 used in the meta-analysis. Confidence intervals from all 87 base papers were converted to <i>p</i>-values. <i>p</i>-value plots for each air component were constructed to evaluate the effect heterogeneity of the <i>p</i>-values. Results: The number of statistical tests possible in the 17 selected base papers was large, median = 15,360 (interquartile range = 1536–40,960), in comparison to results presented. Each <i>p</i>-value plot showed a two-component mixture with small <i>p</i>-values < 0.001 while other <i>p</i>-values appeared random (<i>p</i>-values > 0.05). Given potentially large numbers of statistical tests conducted in the 17 selected base papers, p-hacking cannot be ruled out as explanations for small <i>p</i>-values. Conclusions: Our interpretation of the meta-analysis is that random <i>p</i>-values indicating null associations are more plausible and the meta-analysis is unlikely to replicate in the absence of bias. |
| first_indexed | 2024-03-10T07:32:11Z |
| format | Article |
| id | doaj.art-a913bba16fe94bc889c038f6100b50e3 |
| institution | Directory Open Access Journal |
| issn | 2673-527X |
| language | English |
| last_indexed | 2024-03-10T07:32:11Z |
| publishDate | 2021-06-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Journal of Respiration |
| spelling | doaj.art-a913bba16fe94bc889c038f6100b50e32023-11-22T13:49:26ZengMDPI AGJournal of Respiration2673-527X2021-06-011317319610.3390/jor1030017Evaluation of a Meta-Analysis of Ambient Air Quality as a Risk Factor for Asthma ExacerbationWarren Kindzierski0Stanley Young1Terry Meyer2John Dunn3School of Public Health, University of Alberta, Edmonton, AB T6G 1C9, CanadaCGStat, Raleigh, NC 27607, USAOutcome Based Medicine, Raleigh, NC 27614, USA401 Rocky Hill Road, Brownwood, TX 76801-0986, USABackground: An irreproducibility crisis currently afflicts a wide range of scientific disciplines, including public health and biomedical science. A study was undertaken to assess the reliability of a meta-analysis examining whether air quality components (carbon monoxide, particulate matter 10 µm and 2.5 µm (PM10 and PM2.5), sulfur dioxide, nitrogen dioxide and ozone) are risk factors for asthma exacerbation. Methods: The number of statistical tests and models were counted in 17 randomly selected base papers from 87 used in the meta-analysis. Confidence intervals from all 87 base papers were converted to <i>p</i>-values. <i>p</i>-value plots for each air component were constructed to evaluate the effect heterogeneity of the <i>p</i>-values. Results: The number of statistical tests possible in the 17 selected base papers was large, median = 15,360 (interquartile range = 1536–40,960), in comparison to results presented. Each <i>p</i>-value plot showed a two-component mixture with small <i>p</i>-values < 0.001 while other <i>p</i>-values appeared random (<i>p</i>-values > 0.05). Given potentially large numbers of statistical tests conducted in the 17 selected base papers, p-hacking cannot be ruled out as explanations for small <i>p</i>-values. Conclusions: Our interpretation of the meta-analysis is that random <i>p</i>-values indicating null associations are more plausible and the meta-analysis is unlikely to replicate in the absence of bias.https://www.mdpi.com/2673-527X/1/3/17air qualityasthmameta-analysis<i>p</i>-value plotsfalse-positivesp-hacking |
| spellingShingle | Warren Kindzierski Stanley Young Terry Meyer John Dunn Evaluation of a Meta-Analysis of Ambient Air Quality as a Risk Factor for Asthma Exacerbation Journal of Respiration air quality asthma meta-analysis <i>p</i>-value plots false-positives p-hacking |
| title | Evaluation of a Meta-Analysis of Ambient Air Quality as a Risk Factor for Asthma Exacerbation |
| title_full | Evaluation of a Meta-Analysis of Ambient Air Quality as a Risk Factor for Asthma Exacerbation |
| title_fullStr | Evaluation of a Meta-Analysis of Ambient Air Quality as a Risk Factor for Asthma Exacerbation |
| title_full_unstemmed | Evaluation of a Meta-Analysis of Ambient Air Quality as a Risk Factor for Asthma Exacerbation |
| title_short | Evaluation of a Meta-Analysis of Ambient Air Quality as a Risk Factor for Asthma Exacerbation |
| title_sort | evaluation of a meta analysis of ambient air quality as a risk factor for asthma exacerbation |
| topic | air quality asthma meta-analysis <i>p</i>-value plots false-positives p-hacking |
| url | https://www.mdpi.com/2673-527X/1/3/17 |
| work_keys_str_mv | AT warrenkindzierski evaluationofametaanalysisofambientairqualityasariskfactorforasthmaexacerbation AT stanleyyoung evaluationofametaanalysisofambientairqualityasariskfactorforasthmaexacerbation AT terrymeyer evaluationofametaanalysisofambientairqualityasariskfactorforasthmaexacerbation AT johndunn evaluationofametaanalysisofambientairqualityasariskfactorforasthmaexacerbation |