The Power of Optimization Over Randomization in Designing Experiments Involving Small Samples
Random assignment, typically seen as the standard in controlled trials, aims to make experimental groups statistically equivalent before treatment. However, with a small sample, which is a practical reality in many disciplines, randomized groups are often too dissimilar to be useful. We propose an a...
| Main Authors: | , , |
|---|---|
| Other Authors: | |
| Format: | Article |
| Language: | en_US |
| Published: |
Institute for Operations Research and the Management Sciences (INFORMS)
2015
|
| Online Access: | http://hdl.handle.net/1721.1/98509 https://orcid.org/0000-0003-1672-0507 https://orcid.org/0000-0002-1985-1003 |
| _version_ | 1811085167661416448 |
|---|---|
| author | Johnson, Mac Kallus, Nathan Bertsimas, Dimitris J |
| author2 | Massachusetts Institute of Technology. Operations Research Center |
| author_facet | Massachusetts Institute of Technology. Operations Research Center Johnson, Mac Kallus, Nathan Bertsimas, Dimitris J |
| author_sort | Johnson, Mac |
| collection | MIT |
| description | Random assignment, typically seen as the standard in controlled trials, aims to make experimental groups statistically equivalent before treatment. However, with a small sample, which is a practical reality in many disciplines, randomized groups are often too dissimilar to be useful. We propose an approach based on discrete linear optimization to create groups whose discrepancy in their means and variances is several orders of magnitude smaller than with randomization. We provide theoretical and computational evidence that groups created by optimization have exponentially lower discrepancy than those created by randomization and that this allows for more powerful statistical inference. |
| first_indexed | 2024-09-23T13:03:56Z |
| format | Article |
| id | mit-1721.1/98509 |
| institution | Massachusetts Institute of Technology |
| language | en_US |
| last_indexed | 2024-09-23T13:03:56Z |
| publishDate | 2015 |
| publisher | Institute for Operations Research and the Management Sciences (INFORMS) |
| record_format | dspace |
| spelling | mit-1721.1/985092023-03-01T02:14:20Z The Power of Optimization Over Randomization in Designing Experiments Involving Small Samples Johnson, Mac Kallus, Nathan Bertsimas, Dimitris J Massachusetts Institute of Technology. Operations Research Center Sloan School of Management Bertsimas, Dimitris J. Johnson, Mac Kallus, Nathan Random assignment, typically seen as the standard in controlled trials, aims to make experimental groups statistically equivalent before treatment. However, with a small sample, which is a practical reality in many disciplines, randomized groups are often too dissimilar to be useful. We propose an approach based on discrete linear optimization to create groups whose discrepancy in their means and variances is several orders of magnitude smaller than with randomization. We provide theoretical and computational evidence that groups created by optimization have exponentially lower discrepancy than those created by randomization and that this allows for more powerful statistical inference. National Science Foundation (U.S.). Graduate Research Fellowship (Grant 1122374) 2015-09-15T16:52:53Z 2015-09-15T16:52:53Z 2015-04 2014-09 Article http://purl.org/eprint/type/JournalArticle 0030-364X 1526-5463 http://hdl.handle.net/1721.1/98509 Bertsimas, Dimitris, Mac Johnson, and Nathan Kallus. “The Power of Optimization Over Randomization in Designing Experiments Involving Small Samples.” Operations Research 63, no. 4 (August 2015): 868–876. https://orcid.org/0000-0003-1672-0507 https://orcid.org/0000-0002-1985-1003 en_US http://dx.doi.org/10.1287/opre.2015.1361 Operations Research Creative Commons Attribution-Noncommercial-Share Alike http://creativecommons.org/licenses/by-nc-sa/4.0/ application/pdf Institute for Operations Research and the Management Sciences (INFORMS) MIT web domain |
| spellingShingle | Johnson, Mac Kallus, Nathan Bertsimas, Dimitris J The Power of Optimization Over Randomization in Designing Experiments Involving Small Samples |
| title | The Power of Optimization Over Randomization in Designing Experiments Involving Small Samples |
| title_full | The Power of Optimization Over Randomization in Designing Experiments Involving Small Samples |
| title_fullStr | The Power of Optimization Over Randomization in Designing Experiments Involving Small Samples |
| title_full_unstemmed | The Power of Optimization Over Randomization in Designing Experiments Involving Small Samples |
| title_short | The Power of Optimization Over Randomization in Designing Experiments Involving Small Samples |
| title_sort | power of optimization over randomization in designing experiments involving small samples |
| url | http://hdl.handle.net/1721.1/98509 https://orcid.org/0000-0003-1672-0507 https://orcid.org/0000-0002-1985-1003 |
| work_keys_str_mv | AT johnsonmac thepowerofoptimizationoverrandomizationindesigningexperimentsinvolvingsmallsamples AT kallusnathan thepowerofoptimizationoverrandomizationindesigningexperimentsinvolvingsmallsamples AT bertsimasdimitrisj thepowerofoptimizationoverrandomizationindesigningexperimentsinvolvingsmallsamples AT johnsonmac powerofoptimizationoverrandomizationindesigningexperimentsinvolvingsmallsamples AT kallusnathan powerofoptimizationoverrandomizationindesigningexperimentsinvolvingsmallsamples AT bertsimasdimitrisj powerofoptimizationoverrandomizationindesigningexperimentsinvolvingsmallsamples |