Robust Inferences from a Before-and-After Study with Multiple Unaffected Control Groups
The before-and-after study with multiple unaffected control groups is widely applied to study treatment effects. The current methods usually assume that the control groups’ differences between the before and after periods, i.e. the group time effects, follow a normal distribution. However, there is...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
De Gruyter
2013-06-01
|
| Series: | Journal of Causal Inference |
| Subjects: | |
| Online Access: | https://doi.org/10.1515/jci-2012-0010 |
| _version_ | 1830192129848115200 |
|---|---|
| author | Wang Pengyuan Traskin Mikhail Small Dylan S. |
| author_facet | Wang Pengyuan Traskin Mikhail Small Dylan S. |
| author_sort | Wang Pengyuan |
| collection | DOAJ |
| description | The before-and-after study with multiple unaffected control groups is widely applied to study treatment effects. The current methods usually assume that the control groups’ differences between the before and after periods, i.e. the group time effects, follow a normal distribution. However, there is usually no strong a priori evidence for the normality assumption, and there are not enough control groups to check the assumption. We propose to use a flexible skew-t distribution family to model group time effects, and consider a range of plausible skew-t distributions. Based on the skew-t distribution assumption, we propose a robust-t method to guarantee nominal significance level under a wide range of skew-t distributions, and hence make the inference robust to misspecification of the distribution of group time effects. We also propose a two-stage approach, which has lower power compared to the robust-t method, but provides an opportunity to conduct sensitivity analysis. Hence, the overall method of analysis is to use the robust-t method to test for the overall hypothesized range of shapes of group variation; if the test fails to reject, use the two-stage method to conduct a sensitivity analysis to see if there is a subset of group variation parameters for which we can be confident that there is a treatment effect. We apply the proposed methods to two datasets. One dataset is from the Current Population Survey (CPS) to study the impact of the Mariel Boatlift on Miami unemployment rates between 1979 and 1982.The other dataset contains the student enrollment and grade repeating data in West Germany in the 1960s with which we study the impact of the short school year in 1966–1967 on grade repeating rates. |
| first_indexed | 2024-12-17T23:51:13Z |
| format | Article |
| id | doaj.art-b23d09bd9ed64917848e2aa3711898a4 |
| institution | Directory Open Access Journal |
| issn | 2193-3677 2193-3685 |
| language | English |
| last_indexed | 2024-12-17T23:51:13Z |
| publishDate | 2013-06-01 |
| publisher | De Gruyter |
| record_format | Article |
| series | Journal of Causal Inference |
| spelling | doaj.art-b23d09bd9ed64917848e2aa3711898a42022-12-21T21:28:12ZengDe GruyterJournal of Causal Inference2193-36772193-36852013-06-011220923410.1515/jci-2012-0010Robust Inferences from a Before-and-After Study with Multiple Unaffected Control GroupsWang Pengyuan0Traskin Mikhail1Small Dylan S.2Yahoo! Lab, 701 1st Ave, Sunnyvale, CA 94085, USAAmazon.com, 207 Boren Ave. N., Seattle, WA 98109Department of Statistics, University of Pennsylvania, 3730 Walnut Street 400 Jon M. Huntsman Hall, Philadelphia, PA 19104, USAThe before-and-after study with multiple unaffected control groups is widely applied to study treatment effects. The current methods usually assume that the control groups’ differences between the before and after periods, i.e. the group time effects, follow a normal distribution. However, there is usually no strong a priori evidence for the normality assumption, and there are not enough control groups to check the assumption. We propose to use a flexible skew-t distribution family to model group time effects, and consider a range of plausible skew-t distributions. Based on the skew-t distribution assumption, we propose a robust-t method to guarantee nominal significance level under a wide range of skew-t distributions, and hence make the inference robust to misspecification of the distribution of group time effects. We also propose a two-stage approach, which has lower power compared to the robust-t method, but provides an opportunity to conduct sensitivity analysis. Hence, the overall method of analysis is to use the robust-t method to test for the overall hypothesized range of shapes of group variation; if the test fails to reject, use the two-stage method to conduct a sensitivity analysis to see if there is a subset of group variation parameters for which we can be confident that there is a treatment effect. We apply the proposed methods to two datasets. One dataset is from the Current Population Survey (CPS) to study the impact of the Mariel Boatlift on Miami unemployment rates between 1979 and 1982.The other dataset contains the student enrollment and grade repeating data in West Germany in the 1960s with which we study the impact of the short school year in 1966–1967 on grade repeating rates.https://doi.org/10.1515/jci-2012-0010before-and-after studydifference-in-differencerobust inferenceskew-t distributiongroup time effect |
| spellingShingle | Wang Pengyuan Traskin Mikhail Small Dylan S. Robust Inferences from a Before-and-After Study with Multiple Unaffected Control Groups Journal of Causal Inference before-and-after study difference-in-difference robust inference skew-t distribution group time effect |
| title | Robust Inferences from a Before-and-After Study with Multiple Unaffected Control Groups |
| title_full | Robust Inferences from a Before-and-After Study with Multiple Unaffected Control Groups |
| title_fullStr | Robust Inferences from a Before-and-After Study with Multiple Unaffected Control Groups |
| title_full_unstemmed | Robust Inferences from a Before-and-After Study with Multiple Unaffected Control Groups |
| title_short | Robust Inferences from a Before-and-After Study with Multiple Unaffected Control Groups |
| title_sort | robust inferences from a before and after study with multiple unaffected control groups |
| topic | before-and-after study difference-in-difference robust inference skew-t distribution group time effect |
| url | https://doi.org/10.1515/jci-2012-0010 |
| work_keys_str_mv | AT wangpengyuan robustinferencesfromabeforeandafterstudywithmultipleunaffectedcontrolgroups AT traskinmikhail robustinferencesfromabeforeandafterstudywithmultipleunaffectedcontrolgroups AT smalldylans robustinferencesfromabeforeandafterstudywithmultipleunaffectedcontrolgroups |