Towards a pragmatic use of statistics in ecology
Although null hypothesis testing (NHT) is the primary method for analyzing data in many natural sciences, it has been increasingly criticized. Recently, approaches based on information theory (IT) have become popular and were held by many to be superior because it enables researchers to properly ass...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
PeerJ Inc.
2021-09-01
|
| Series: | PeerJ |
| Subjects: | |
| Online Access: | https://peerj.com/articles/12090.pdf |
| _version_ | 1797417675837669376 |
|---|---|
| author | Leonardo Braga Castilho Paulo Inácio Prado |
| author_facet | Leonardo Braga Castilho Paulo Inácio Prado |
| author_sort | Leonardo Braga Castilho |
| collection | DOAJ |
| description | Although null hypothesis testing (NHT) is the primary method for analyzing data in many natural sciences, it has been increasingly criticized. Recently, approaches based on information theory (IT) have become popular and were held by many to be superior because it enables researchers to properly assess the strength of the evidence that data provide for competing hypotheses. Many studies have compared IT and NHT in the context of model selection and stepwise regression, but a systematic comparison of the most basic uses of statistics by ecologists is still lacking. We used computer simulations to compare how both approaches perform in four basic test designs (t-test, ANOVA, correlation tests, and multiple linear regression). Performance was measured by the proportion of simulated samples for which each method provided the correct conclusion (power), the proportion of detected effects with a wrong sign (S-error), and the mean ratio of the estimated effect to the true effect (M-error). We also checked if the p-value from significance tests correlated to a measure of strength of evidence, the Akaike weight. In general both methods performed equally well. The concordance is explained by the monotonic relationship between p-values and evidence weights in simple designs, which agree with analytic results. Our results show that researchers can agree on the conclusions drawn from a data set even when they are using different statistical approaches. By focusing on the practical consequences of inferences, such a pragmatic view of statistics can promote insightful dialogue among researchers on how to find a common ground from different pieces of evidence. A less dogmatic view of statistical inference can also help to broaden the debate about the role of statistics in science to the entire path that leads from a research hypothesis to a statistical hypothesis. |
| first_indexed | 2024-03-09T06:22:04Z |
| format | Article |
| id | doaj.art-a671822f97af41c9b63ce467108caff1 |
| institution | Directory Open Access Journal |
| issn | 2167-8359 |
| language | English |
| last_indexed | 2024-03-09T06:22:04Z |
| publishDate | 2021-09-01 |
| publisher | PeerJ Inc. |
| record_format | Article |
| series | PeerJ |
| spelling | doaj.art-a671822f97af41c9b63ce467108caff12023-12-03T11:35:52ZengPeerJ Inc.PeerJ2167-83592021-09-019e1209010.7717/peerj.12090Towards a pragmatic use of statistics in ecologyLeonardo Braga Castilho0Paulo Inácio Prado1Universidade de Brasília, Brasília, BrazilDepartamento de Ecologia/Instituto de Biociências, Universidade de São Paulo, São Paulo, São Paulo, BrazilAlthough null hypothesis testing (NHT) is the primary method for analyzing data in many natural sciences, it has been increasingly criticized. Recently, approaches based on information theory (IT) have become popular and were held by many to be superior because it enables researchers to properly assess the strength of the evidence that data provide for competing hypotheses. Many studies have compared IT and NHT in the context of model selection and stepwise regression, but a systematic comparison of the most basic uses of statistics by ecologists is still lacking. We used computer simulations to compare how both approaches perform in four basic test designs (t-test, ANOVA, correlation tests, and multiple linear regression). Performance was measured by the proportion of simulated samples for which each method provided the correct conclusion (power), the proportion of detected effects with a wrong sign (S-error), and the mean ratio of the estimated effect to the true effect (M-error). We also checked if the p-value from significance tests correlated to a measure of strength of evidence, the Akaike weight. In general both methods performed equally well. The concordance is explained by the monotonic relationship between p-values and evidence weights in simple designs, which agree with analytic results. Our results show that researchers can agree on the conclusions drawn from a data set even when they are using different statistical approaches. By focusing on the practical consequences of inferences, such a pragmatic view of statistics can promote insightful dialogue among researchers on how to find a common ground from different pieces of evidence. A less dogmatic view of statistical inference can also help to broaden the debate about the role of statistics in science to the entire path that leads from a research hypothesis to a statistical hypothesis.https://peerj.com/articles/12090.pdfAICLikelihoodM-errorModel-based inferenceNull hypothesis testingPower |
| spellingShingle | Leonardo Braga Castilho Paulo Inácio Prado Towards a pragmatic use of statistics in ecology PeerJ AIC Likelihood M-error Model-based inference Null hypothesis testing Power |
| title | Towards a pragmatic use of statistics in ecology |
| title_full | Towards a pragmatic use of statistics in ecology |
| title_fullStr | Towards a pragmatic use of statistics in ecology |
| title_full_unstemmed | Towards a pragmatic use of statistics in ecology |
| title_short | Towards a pragmatic use of statistics in ecology |
| title_sort | towards a pragmatic use of statistics in ecology |
| topic | AIC Likelihood M-error Model-based inference Null hypothesis testing Power |
| url | https://peerj.com/articles/12090.pdf |
| work_keys_str_mv | AT leonardobragacastilho towardsapragmaticuseofstatisticsinecology AT pauloinacioprado towardsapragmaticuseofstatisticsinecology |