Elastic-net regularization in learning theory
Within the framework of statistical learning theory we analyze in detail the so-called elastic-net regularization scheme proposed by Zou and Hastie [H. Zou, T. Hastie, Regularization and variable selection via the elastic net, J. R. Stat. Soc. Ser. B, 67(2) (2005) 301–320] for the selection of group...
| Main Authors: | , , |
|---|---|
| Other Authors: | |
| Format: | Article |
| Language: | en_US |
| Published: |
Elsevier
2015
|
| Online Access: | http://hdl.handle.net/1721.1/96186 https://orcid.org/0000-0001-6376-4786 |
| _version_ | 1826212746971054080 |
|---|---|
| author | De Mol, Christine De Vito, Ernesto Rosasco, Lorenzo Andrea |
| author2 | Massachusetts Institute of Technology. Center for Biological & Computational Learning |
| author_facet | Massachusetts Institute of Technology. Center for Biological & Computational Learning De Mol, Christine De Vito, Ernesto Rosasco, Lorenzo Andrea |
| author_sort | De Mol, Christine |
| collection | MIT |
| description | Within the framework of statistical learning theory we analyze in detail the so-called elastic-net regularization scheme proposed by Zou and Hastie [H. Zou, T. Hastie, Regularization and variable selection via the elastic net, J. R. Stat. Soc. Ser. B, 67(2) (2005) 301–320] for the selection of groups of correlated variables. To investigate the statistical properties of this scheme and in particular its consistency properties, we set up a suitable mathematical framework. Our setting is random-design regression where we allow the response variable to be vector-valued and we consider prediction functions which are linear combinations of elements (features) in an infinite-dimensional dictionary. Under the assumption that the regression function admits a sparse representation on the dictionary, we prove that there exists a particular “elastic-net representation” of the regression function such that, if the number of data increases, the elastic-net estimator is consistent not only for prediction but also for variable/feature selection. Our results include finite-sample bounds and an adaptive scheme to select the regularization parameter. Moreover, using convex analysis tools, we derive an iterative thresholding algorithm for computing the elastic-net solution which is different from the optimization procedure originally proposed in the above-cited work. |
| first_indexed | 2024-09-23T15:37:35Z |
| format | Article |
| id | mit-1721.1/96186 |
| institution | Massachusetts Institute of Technology |
| language | en_US |
| last_indexed | 2024-09-23T15:37:35Z |
| publishDate | 2015 |
| publisher | Elsevier |
| record_format | dspace |
| spelling | mit-1721.1/961862022-10-02T02:58:36Z Elastic-net regularization in learning theory De Mol, Christine De Vito, Ernesto Rosasco, Lorenzo Andrea Massachusetts Institute of Technology. Center for Biological & Computational Learning McGovern Institute for Brain Research at MIT Rosasco, Lorenzo Andrea Within the framework of statistical learning theory we analyze in detail the so-called elastic-net regularization scheme proposed by Zou and Hastie [H. Zou, T. Hastie, Regularization and variable selection via the elastic net, J. R. Stat. Soc. Ser. B, 67(2) (2005) 301–320] for the selection of groups of correlated variables. To investigate the statistical properties of this scheme and in particular its consistency properties, we set up a suitable mathematical framework. Our setting is random-design regression where we allow the response variable to be vector-valued and we consider prediction functions which are linear combinations of elements (features) in an infinite-dimensional dictionary. Under the assumption that the regression function admits a sparse representation on the dictionary, we prove that there exists a particular “elastic-net representation” of the regression function such that, if the number of data increases, the elastic-net estimator is consistent not only for prediction but also for variable/feature selection. Our results include finite-sample bounds and an adaptive scheme to select the regularization parameter. Moreover, using convex analysis tools, we derive an iterative thresholding algorithm for computing the elastic-net solution which is different from the optimization procedure originally proposed in the above-cited work. European Union. Integrated Project Health-e-Child (IST-2004-027749) Italy. Ministry of Education, Universities and Research. FIRB Project (RBIN04PARL) 2015-03-25T17:32:22Z 2015-03-25T17:32:22Z 2009-01 2008-08 Article http://purl.org/eprint/type/JournalArticle 0885064X 1090-2708 http://hdl.handle.net/1721.1/96186 De Mol, Christine, Ernesto De Vito, and Lorenzo Rosasco. “Elastic-Net Regularization in Learning Theory.” Journal of Complexity 25, no. 2 (April 2009): 201–230. © 2009 Elsevier Inc. https://orcid.org/0000-0001-6376-4786 en_US http://dx.doi.org/10.1016/j.jco.2009.01.002 Journal of Complexity Article is made available in accordance with the publisher's policy and may be subject to US copyright law. Please refer to the publisher's site for terms of use. application/pdf Elsevier Elsevier |
| spellingShingle | De Mol, Christine De Vito, Ernesto Rosasco, Lorenzo Andrea Elastic-net regularization in learning theory |
| title | Elastic-net regularization in learning theory |
| title_full | Elastic-net regularization in learning theory |
| title_fullStr | Elastic-net regularization in learning theory |
| title_full_unstemmed | Elastic-net regularization in learning theory |
| title_short | Elastic-net regularization in learning theory |
| title_sort | elastic net regularization in learning theory |
| url | http://hdl.handle.net/1721.1/96186 https://orcid.org/0000-0001-6376-4786 |
| work_keys_str_mv | AT demolchristine elasticnetregularizationinlearningtheory AT devitoernesto elasticnetregularizationinlearningtheory AT rosascolorenzoandrea elasticnetregularizationinlearningtheory |