Conditional as-if analyses in randomized experiments
The injunction to “analyze the way you randomize” is well known to statisticians since Fisher advocated for randomization as the basis of inference. Yet even those convinced by the merits of randomization-based inference seldom follow this injunction to the letter. Bernoulli randomized experiments a...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
De Gruyter
2021-12-01
|
| Series: | Journal of Causal Inference |
| Subjects: | |
| Online Access: | https://doi.org/10.1515/jci-2021-0012 |
| _version_ | 1798030171494678528 |
|---|---|
| author | Pashley Nicole E. Basse Guillaume W. Miratrix Luke W. |
| author_facet | Pashley Nicole E. Basse Guillaume W. Miratrix Luke W. |
| author_sort | Pashley Nicole E. |
| collection | DOAJ |
| description | The injunction to “analyze the way you randomize” is well known to statisticians since Fisher advocated for randomization as the basis of inference. Yet even those convinced by the merits of randomization-based inference seldom follow this injunction to the letter. Bernoulli randomized experiments are often analyzed as completely randomized experiments, and completely randomized experiments are analyzed as if they had been stratified; more generally, it is not uncommon to analyze an experiment as if it had been randomized differently. This article examines the theoretical foundation behind this practice within a randomization-based framework. Specifically, we ask when is it legitimate to analyze an experiment randomized according to one design as if it had been randomized according to some other design. We show that a sufficient condition for this type of analysis to be valid is that the design used for analysis should be derived from the original design by an appropriate form of conditioning. We use our theory to justify certain existing methods, question others, and finally suggest new methodological insights such as conditioning on approximate covariate balance. |
| first_indexed | 2024-04-11T19:37:00Z |
| format | Article |
| id | doaj.art-1d0c28b3add04910b9a2c5da2ccb68b8 |
| institution | Directory Open Access Journal |
| issn | 2193-3685 |
| language | English |
| last_indexed | 2024-04-11T19:37:00Z |
| publishDate | 2021-12-01 |
| publisher | De Gruyter |
| record_format | Article |
| series | Journal of Causal Inference |
| spelling | doaj.art-1d0c28b3add04910b9a2c5da2ccb68b82022-12-22T04:06:49ZengDe GruyterJournal of Causal Inference2193-36852021-12-019126428410.1515/jci-2021-0012Conditional as-if analyses in randomized experimentsPashley Nicole E.0Basse Guillaume W.1Miratrix Luke W.2Department of Statistics, Rutgers University, Piscataway, NJ 08854, United States of AmericaDepartment of Statistics, Stanford University, Stanford, CA 94305, United States of AmericaHarvard Graduate School of Education, Cambridge, MA 02138, United States of AmericaThe injunction to “analyze the way you randomize” is well known to statisticians since Fisher advocated for randomization as the basis of inference. Yet even those convinced by the merits of randomization-based inference seldom follow this injunction to the letter. Bernoulli randomized experiments are often analyzed as completely randomized experiments, and completely randomized experiments are analyzed as if they had been stratified; more generally, it is not uncommon to analyze an experiment as if it had been randomized differently. This article examines the theoretical foundation behind this practice within a randomization-based framework. Specifically, we ask when is it legitimate to analyze an experiment randomized according to one design as if it had been randomized according to some other design. We show that a sufficient condition for this type of analysis to be valid is that the design used for analysis should be derived from the original design by an appropriate form of conditioning. We use our theory to justify certain existing methods, question others, and finally suggest new methodological insights such as conditioning on approximate covariate balance.https://doi.org/10.1515/jci-2021-0012ancillary statisticscausal inferenceconditional inferencerandomization inferencerelevance62d9962k99 |
| spellingShingle | Pashley Nicole E. Basse Guillaume W. Miratrix Luke W. Conditional as-if analyses in randomized experiments Journal of Causal Inference ancillary statistics causal inference conditional inference randomization inference relevance 62d99 62k99 |
| title | Conditional as-if analyses in randomized experiments |
| title_full | Conditional as-if analyses in randomized experiments |
| title_fullStr | Conditional as-if analyses in randomized experiments |
| title_full_unstemmed | Conditional as-if analyses in randomized experiments |
| title_short | Conditional as-if analyses in randomized experiments |
| title_sort | conditional as if analyses in randomized experiments |
| topic | ancillary statistics causal inference conditional inference randomization inference relevance 62d99 62k99 |
| url | https://doi.org/10.1515/jci-2021-0012 |
| work_keys_str_mv | AT pashleynicolee conditionalasifanalysesinrandomizedexperiments AT basseguillaumew conditionalasifanalysesinrandomizedexperiments AT miratrixlukew conditionalasifanalysesinrandomizedexperiments |