Subset Selection with Shrinkage: Sparse Linear Modeling When the SNR Is Low
<jats:p> Learning Compact High-Dimensional Models in Noisy Environments </jats:p><jats:p> Building compact, interpretable statistical models where the output depends upon a small number of input features is a well-known problem in modern analytics applications. A fundamental tool u...
| Main Authors: | , , |
|---|---|
| Other Authors: | |
| Format: | Article |
| Language: | English |
| Published: |
Institute for Operations Research and the Management Sciences (INFORMS)
2022
|
| Online Access: | https://hdl.handle.net/1721.1/144220 |
| _version_ | 1826211866958888960 |
|---|---|
| author | Mazumder, Rahul Radchenko, Peter Dedieu, Antoine |
| author2 | Sloan School of Management |
| author_facet | Sloan School of Management Mazumder, Rahul Radchenko, Peter Dedieu, Antoine |
| author_sort | Mazumder, Rahul |
| collection | MIT |
| description | <jats:p> Learning Compact High-Dimensional Models in Noisy Environments </jats:p><jats:p> Building compact, interpretable statistical models where the output depends upon a small number of input features is a well-known problem in modern analytics applications. A fundamental tool used in this context is the prominent best subset selection (BSS) procedure, which seeks to obtain the best linear fit to data subject to a constraint on the number of nonzero features. Whereas the BSS procedure works exceptionally well in some regimes, it performs pretty poorly in out-of-sample predictive performance when the underlying data are noisy, which is quite common in practice. In this paper, we explore this relatively less-understood overfitting behavior of BSS in low-signal noisy environments and propose alternatives that appear to mitigate such shortcomings. We study the theoretical statistical properties of our proposed regularized BSS procedure and show promising computational results on various data sets, using tools from integer programming and first-order methods. </jats:p> |
| first_indexed | 2024-09-23T15:12:40Z |
| format | Article |
| id | mit-1721.1/144220 |
| institution | Massachusetts Institute of Technology |
| language | English |
| last_indexed | 2024-09-23T15:12:40Z |
| publishDate | 2022 |
| publisher | Institute for Operations Research and the Management Sciences (INFORMS) |
| record_format | dspace |
| spelling | mit-1721.1/1442202023-08-11T18:19:08Z Subset Selection with Shrinkage: Sparse Linear Modeling When the SNR Is Low Mazumder, Rahul Radchenko, Peter Dedieu, Antoine Sloan School of Management <jats:p> Learning Compact High-Dimensional Models in Noisy Environments </jats:p><jats:p> Building compact, interpretable statistical models where the output depends upon a small number of input features is a well-known problem in modern analytics applications. A fundamental tool used in this context is the prominent best subset selection (BSS) procedure, which seeks to obtain the best linear fit to data subject to a constraint on the number of nonzero features. Whereas the BSS procedure works exceptionally well in some regimes, it performs pretty poorly in out-of-sample predictive performance when the underlying data are noisy, which is quite common in practice. In this paper, we explore this relatively less-understood overfitting behavior of BSS in low-signal noisy environments and propose alternatives that appear to mitigate such shortcomings. We study the theoretical statistical properties of our proposed regularized BSS procedure and show promising computational results on various data sets, using tools from integer programming and first-order methods. </jats:p> 2022-08-04T15:30:43Z 2022-08-04T15:30:43Z 2022-05-24 2022-08-04T15:19:28Z Article http://purl.org/eprint/type/JournalArticle https://hdl.handle.net/1721.1/144220 Mazumder, Rahul, Radchenko, Peter and Dedieu, Antoine. 2022. "Subset Selection with Shrinkage: Sparse Linear Modeling When the SNR Is Low." Operations Research. en 10.1287/opre.2022.2276 Operations Research Creative Commons Attribution-Noncommercial-Share Alike http://creativecommons.org/licenses/by-nc-sa/4.0/ application/pdf Institute for Operations Research and the Management Sciences (INFORMS) arXiv |
| spellingShingle | Mazumder, Rahul Radchenko, Peter Dedieu, Antoine Subset Selection with Shrinkage: Sparse Linear Modeling When the SNR Is Low |
| title | Subset Selection with Shrinkage: Sparse Linear Modeling When the SNR Is Low |
| title_full | Subset Selection with Shrinkage: Sparse Linear Modeling When the SNR Is Low |
| title_fullStr | Subset Selection with Shrinkage: Sparse Linear Modeling When the SNR Is Low |
| title_full_unstemmed | Subset Selection with Shrinkage: Sparse Linear Modeling When the SNR Is Low |
| title_short | Subset Selection with Shrinkage: Sparse Linear Modeling When the SNR Is Low |
| title_sort | subset selection with shrinkage sparse linear modeling when the snr is low |
| url | https://hdl.handle.net/1721.1/144220 |
| work_keys_str_mv | AT mazumderrahul subsetselectionwithshrinkagesparselinearmodelingwhenthesnrislow AT radchenkopeter subsetselectionwithshrinkagesparselinearmodelingwhenthesnrislow AT dedieuantoine subsetselectionwithshrinkagesparselinearmodelingwhenthesnrislow |