Setting an optimal α that minimizes errors in null hypothesis significance tests.
Null hypothesis significance testing has been under attack in recent years, partly owing to the arbitrary nature of setting α (the decision-making threshold and probability of Type I error) at a constant value, usually 0.05. If the goal of null hypothesis testing is to present conclusions in which w...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Public Library of Science (PLoS)
2012-01-01
|
| Series: | PLoS ONE |
| Online Access: | https://www.ncbi.nlm.nih.gov/pmc/articles/pmid/22389720/pdf/?tool=EBI |
| _version_ | 1818736031126519808 |
|---|---|
| author | Joseph F Mudge Leanne F Baker Christopher B Edge Jeff E Houlahan |
| author_facet | Joseph F Mudge Leanne F Baker Christopher B Edge Jeff E Houlahan |
| author_sort | Joseph F Mudge |
| collection | DOAJ |
| description | Null hypothesis significance testing has been under attack in recent years, partly owing to the arbitrary nature of setting α (the decision-making threshold and probability of Type I error) at a constant value, usually 0.05. If the goal of null hypothesis testing is to present conclusions in which we have the highest possible confidence, then the only logical decision-making threshold is the value that minimizes the probability (or occasionally, cost) of making errors. Setting α to minimize the combination of Type I and Type II error at a critical effect size can easily be accomplished for traditional statistical tests by calculating the α associated with the minimum average of α and β at the critical effect size. This technique also has the flexibility to incorporate prior probabilities of null and alternate hypotheses and/or relative costs of Type I and Type II errors, if known. Using an optimal α results in stronger scientific inferences because it estimates and minimizes both Type I errors and relevant Type II errors for a test. It also results in greater transparency concerning assumptions about relevant effect size(s) and the relative costs of Type I and II errors. By contrast, the use of α = 0.05 results in arbitrary decisions about what effect sizes will likely be considered significant, if real, and results in arbitrary amounts of Type II error for meaningful potential effect sizes. We cannot identify a rationale for continuing to arbitrarily use α = 0.05 for null hypothesis significance tests in any field, when it is possible to determine an optimal α. |
| first_indexed | 2024-12-18T00:30:41Z |
| format | Article |
| id | doaj.art-aa4cfdbd11174377bacc2dd7e5c8c8fd |
| institution | Directory Open Access Journal |
| issn | 1932-6203 |
| language | English |
| last_indexed | 2024-12-18T00:30:41Z |
| publishDate | 2012-01-01 |
| publisher | Public Library of Science (PLoS) |
| record_format | Article |
| series | PLoS ONE |
| spelling | doaj.art-aa4cfdbd11174377bacc2dd7e5c8c8fd2022-12-21T21:27:08ZengPublic Library of Science (PLoS)PLoS ONE1932-62032012-01-0172e3273410.1371/journal.pone.0032734Setting an optimal α that minimizes errors in null hypothesis significance tests.Joseph F MudgeLeanne F BakerChristopher B EdgeJeff E HoulahanNull hypothesis significance testing has been under attack in recent years, partly owing to the arbitrary nature of setting α (the decision-making threshold and probability of Type I error) at a constant value, usually 0.05. If the goal of null hypothesis testing is to present conclusions in which we have the highest possible confidence, then the only logical decision-making threshold is the value that minimizes the probability (or occasionally, cost) of making errors. Setting α to minimize the combination of Type I and Type II error at a critical effect size can easily be accomplished for traditional statistical tests by calculating the α associated with the minimum average of α and β at the critical effect size. This technique also has the flexibility to incorporate prior probabilities of null and alternate hypotheses and/or relative costs of Type I and Type II errors, if known. Using an optimal α results in stronger scientific inferences because it estimates and minimizes both Type I errors and relevant Type II errors for a test. It also results in greater transparency concerning assumptions about relevant effect size(s) and the relative costs of Type I and II errors. By contrast, the use of α = 0.05 results in arbitrary decisions about what effect sizes will likely be considered significant, if real, and results in arbitrary amounts of Type II error for meaningful potential effect sizes. We cannot identify a rationale for continuing to arbitrarily use α = 0.05 for null hypothesis significance tests in any field, when it is possible to determine an optimal α.https://www.ncbi.nlm.nih.gov/pmc/articles/pmid/22389720/pdf/?tool=EBI |
| spellingShingle | Joseph F Mudge Leanne F Baker Christopher B Edge Jeff E Houlahan Setting an optimal α that minimizes errors in null hypothesis significance tests. PLoS ONE |
| title | Setting an optimal α that minimizes errors in null hypothesis significance tests. |
| title_full | Setting an optimal α that minimizes errors in null hypothesis significance tests. |
| title_fullStr | Setting an optimal α that minimizes errors in null hypothesis significance tests. |
| title_full_unstemmed | Setting an optimal α that minimizes errors in null hypothesis significance tests. |
| title_short | Setting an optimal α that minimizes errors in null hypothesis significance tests. |
| title_sort | setting an optimal α that minimizes errors in null hypothesis significance tests |
| url | https://www.ncbi.nlm.nih.gov/pmc/articles/pmid/22389720/pdf/?tool=EBI |
| work_keys_str_mv | AT josephfmudge settinganoptimalathatminimizeserrorsinnullhypothesissignificancetests AT leannefbaker settinganoptimalathatminimizeserrorsinnullhypothesissignificancetests AT christopherbedge settinganoptimalathatminimizeserrorsinnullhypothesissignificancetests AT jeffehoulahan settinganoptimalathatminimizeserrorsinnullhypothesissignificancetests |