Randomization Tests that Condition on Non-Categorical Covariate Balance
A benefit of randomized experiments is that covariate distributions of treatment and control groups are balanced on average, resulting in simple unbiased estimators for treatment effects. However, it is possible that a particular randomization yields covariate imbalances that researchers want to add...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
De Gruyter
2019-04-01
|
| Series: | Journal of Causal Inference |
| Subjects: | |
| Online Access: | https://doi.org/10.1515/jci-2018-0004 |
| _version_ | 1818736891363590144 |
|---|---|
| author | Branson Zach Miratrix Luke W. |
| author_facet | Branson Zach Miratrix Luke W. |
| author_sort | Branson Zach |
| collection | DOAJ |
| description | A benefit of randomized experiments is that covariate distributions of treatment and control groups are balanced on average, resulting in simple unbiased estimators for treatment effects. However, it is possible that a particular randomization yields covariate imbalances that researchers want to address in the analysis stage through adjustment or other methods. Here we present a randomization test that conditions on covariate balance by only considering treatment assignments that are similar to the observed one in terms of covariate balance. Previous conditional randomization tests have only allowed for categorical covariates, while our randomization test allows for any type of covariate. Through extensive simulation studies, we find that our conditional randomization test is more powerful than unconditional randomization tests and other conditional tests. Furthermore, we find that our conditional randomization test is valid (1) unconditionally across levels of covariate balance, and (2) conditional on particular levels of covariate balance. Meanwhile, unconditional randomization tests are valid for (1) but not (2). Finally, we find that our conditional randomization test is similar to a randomization test that uses a model-adjusted test statistic. |
| first_indexed | 2024-12-18T00:44:21Z |
| format | Article |
| id | doaj.art-ac75976270654c0b8729ef7363856095 |
| institution | Directory Open Access Journal |
| issn | 2193-3677 2193-3685 |
| language | English |
| last_indexed | 2024-12-18T00:44:21Z |
| publishDate | 2019-04-01 |
| publisher | De Gruyter |
| record_format | Article |
| series | Journal of Causal Inference |
| spelling | doaj.art-ac75976270654c0b8729ef73638560952022-12-21T21:26:48ZengDe GruyterJournal of Causal Inference2193-36772193-36852019-04-01711355410.1515/jci-2018-0004Randomization Tests that Condition on Non-Categorical Covariate BalanceBranson Zach0Miratrix Luke W.1Department of Statistics, Harvard University, Cambridge, Massachusetts, United StatesGraduate School of Education and Department of Statistics, Harvard University, Cambridge, Massachusetts, United StatesA benefit of randomized experiments is that covariate distributions of treatment and control groups are balanced on average, resulting in simple unbiased estimators for treatment effects. However, it is possible that a particular randomization yields covariate imbalances that researchers want to address in the analysis stage through adjustment or other methods. Here we present a randomization test that conditions on covariate balance by only considering treatment assignments that are similar to the observed one in terms of covariate balance. Previous conditional randomization tests have only allowed for categorical covariates, while our randomization test allows for any type of covariate. Through extensive simulation studies, we find that our conditional randomization test is more powerful than unconditional randomization tests and other conditional tests. Furthermore, we find that our conditional randomization test is valid (1) unconditionally across levels of covariate balance, and (2) conditional on particular levels of covariate balance. Meanwhile, unconditional randomization tests are valid for (1) but not (2). Finally, we find that our conditional randomization test is similar to a randomization test that uses a model-adjusted test statistic.https://doi.org/10.1515/jci-2018-0004conditional inferencecovariate adjustmentstatistical powervalidity |
| spellingShingle | Branson Zach Miratrix Luke W. Randomization Tests that Condition on Non-Categorical Covariate Balance Journal of Causal Inference conditional inference covariate adjustment statistical power validity |
| title | Randomization Tests that Condition on Non-Categorical Covariate Balance |
| title_full | Randomization Tests that Condition on Non-Categorical Covariate Balance |
| title_fullStr | Randomization Tests that Condition on Non-Categorical Covariate Balance |
| title_full_unstemmed | Randomization Tests that Condition on Non-Categorical Covariate Balance |
| title_short | Randomization Tests that Condition on Non-Categorical Covariate Balance |
| title_sort | randomization tests that condition on non categorical covariate balance |
| topic | conditional inference covariate adjustment statistical power validity |
| url | https://doi.org/10.1515/jci-2018-0004 |
| work_keys_str_mv | AT bransonzach randomizationteststhatconditiononnoncategoricalcovariatebalance AT miratrixlukew randomizationteststhatconditiononnoncategoricalcovariatebalance |