High-Dimensional Methods and Inference on Structural and Treatment Effects
Data with a large number of variables relative to the sample size—"high-dimensional data"—are readily available and increasingly common in empirical economics. High-dimensional data arise through a combination of two phenomena. First, the data may be inherently high dimensional in that man...
| Main Authors: | , , |
|---|---|
| Other Authors: | |
| Format: | Article |
| Language: | en_US |
| Published: |
American Economic Association
2015
|
| Online Access: | http://hdl.handle.net/1721.1/95959 https://orcid.org/0000-0002-3250-6714 |
| _version_ | 1826192175272034304 |
|---|---|
| author | Belloni, Alexandre Hansen, Christian Chernozhukov, Victor V. |
| author2 | Massachusetts Institute of Technology. Department of Economics |
| author_facet | Massachusetts Institute of Technology. Department of Economics Belloni, Alexandre Hansen, Christian Chernozhukov, Victor V. |
| author_sort | Belloni, Alexandre |
| collection | MIT |
| description | Data with a large number of variables relative to the sample size—"high-dimensional data"—are readily available and increasingly common in empirical economics. High-dimensional data arise through a combination of two phenomena. First, the data may be inherently high dimensional in that many different characteristics per observation are available. For example, the US Census collects information on hundreds of individual characteristics and scanner datasets record transaction-level data for households across a wide range of products. Second, even when the number of available variables is relatively small, researchers rarely know the exact functional form with which the small number of variables enter the model of interest. Researchers are thus faced with a large set of potential variables formed by different ways of interacting and transforming the underlying variables. This paper provides an overview of how innovations in "data mining" can be adapted and modified to provide high-quality inference about model parameters. Note that we use the term "data mining" in a modern sense which denotes a principled search for "true" predictive power that guards against false discovery and overfitting, does not erroneously equate in-sample fit to out-of-sample predictive ability, and accurately accounts for using the same data to examine many different hypotheses or models. |
| first_indexed | 2024-09-23T09:07:20Z |
| format | Article |
| id | mit-1721.1/95959 |
| institution | Massachusetts Institute of Technology |
| language | en_US |
| last_indexed | 2024-09-23T09:07:20Z |
| publishDate | 2015 |
| publisher | American Economic Association |
| record_format | dspace |
| spelling | mit-1721.1/959592022-09-30T13:32:25Z High-Dimensional Methods and Inference on Structural and Treatment Effects Belloni, Alexandre Hansen, Christian Chernozhukov, Victor V. Massachusetts Institute of Technology. Department of Economics Chernozhukov, Victor V. Data with a large number of variables relative to the sample size—"high-dimensional data"—are readily available and increasingly common in empirical economics. High-dimensional data arise through a combination of two phenomena. First, the data may be inherently high dimensional in that many different characteristics per observation are available. For example, the US Census collects information on hundreds of individual characteristics and scanner datasets record transaction-level data for households across a wide range of products. Second, even when the number of available variables is relatively small, researchers rarely know the exact functional form with which the small number of variables enter the model of interest. Researchers are thus faced with a large set of potential variables formed by different ways of interacting and transforming the underlying variables. This paper provides an overview of how innovations in "data mining" can be adapted and modified to provide high-quality inference about model parameters. Note that we use the term "data mining" in a modern sense which denotes a principled search for "true" predictive power that guards against false discovery and overfitting, does not erroneously equate in-sample fit to out-of-sample predictive ability, and accurately accounts for using the same data to examine many different hypotheses or models. 2015-03-11T19:13:40Z 2015-03-11T19:13:40Z 2014-05 Article http://purl.org/eprint/type/JournalArticle 0895-3309 1944-7965 http://hdl.handle.net/1721.1/95959 Belloni, Alexandre, Victor Chernozhukov, and Christian Hansen. “ High-Dimensional Methods and Inference on Structural and Treatment Effects † .” Journal of Economic Perspectives 28, no. 2 (May 2014): 29–50. © 2014 American Economic Association https://orcid.org/0000-0002-3250-6714 en_US http://dx.doi.org/10.1257/jep.28.2.29 Journal of Economic Perspectives Article is made available in accordance with the publisher's policy and may be subject to US copyright law. Please refer to the publisher's site for terms of use. application/pdf American Economic Association American Economic Association |
| spellingShingle | Belloni, Alexandre Hansen, Christian Chernozhukov, Victor V. High-Dimensional Methods and Inference on Structural and Treatment Effects |
| title | High-Dimensional Methods and Inference on Structural and Treatment Effects |
| title_full | High-Dimensional Methods and Inference on Structural and Treatment Effects |
| title_fullStr | High-Dimensional Methods and Inference on Structural and Treatment Effects |
| title_full_unstemmed | High-Dimensional Methods and Inference on Structural and Treatment Effects |
| title_short | High-Dimensional Methods and Inference on Structural and Treatment Effects |
| title_sort | high dimensional methods and inference on structural and treatment effects |
| url | http://hdl.handle.net/1721.1/95959 https://orcid.org/0000-0002-3250-6714 |
| work_keys_str_mv | AT bellonialexandre highdimensionalmethodsandinferenceonstructuralandtreatmenteffects AT hansenchristian highdimensionalmethodsandinferenceonstructuralandtreatmenteffects AT chernozhukovvictorv highdimensionalmethodsandinferenceonstructuralandtreatmenteffects |