Characterization of the equivalence of robustification and regularization in linear and matrix regression
The notion of developing statistical methods in machine learning which are robust to adversarial perturbations in the underlying data has been the subject of increasing interest in recent years. A common feature of this work is that the adversarial robustification often corresponds exactly to regula...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Elsevier BV
2021
|
| Online Access: | https://hdl.handle.net/1721.1/135747 |
| _version_ | 1811098334421581824 |
|---|---|
| author | Bertsimas, Dimitris Copenhaver, Martin S |
| author_facet | Bertsimas, Dimitris Copenhaver, Martin S |
| author_sort | Bertsimas, Dimitris |
| collection | MIT |
| description | The notion of developing statistical methods in machine learning which are robust to adversarial perturbations in the underlying data has been the subject of increasing interest in recent years. A common feature of this work is that the adversarial robustification often corresponds exactly to regularization methods which appear as a loss function plus a penalty. In this paper we deepen and extend the understanding of the connection between robustification and regularization (as achieved by penalization) in regression problems. Specifically, (a) In the context of linear regression, we characterize precisely under which conditions on the model of uncertainty used and on the loss function penalties robustification and regularization are equivalent.(b) We extend the characterization of robustification and regularization to matrix regression problems (matrix completion and Principal Component Analysis). |
| first_indexed | 2024-09-23T17:13:26Z |
| format | Article |
| id | mit-1721.1/135747 |
| institution | Massachusetts Institute of Technology |
| language | English |
| last_indexed | 2024-09-23T17:13:26Z |
| publishDate | 2021 |
| publisher | Elsevier BV |
| record_format | dspace |
| spelling | mit-1721.1/1357472021-10-28T03:00:47Z Characterization of the equivalence of robustification and regularization in linear and matrix regression Bertsimas, Dimitris Copenhaver, Martin S The notion of developing statistical methods in machine learning which are robust to adversarial perturbations in the underlying data has been the subject of increasing interest in recent years. A common feature of this work is that the adversarial robustification often corresponds exactly to regularization methods which appear as a loss function plus a penalty. In this paper we deepen and extend the understanding of the connection between robustification and regularization (as achieved by penalization) in regression problems. Specifically, (a) In the context of linear regression, we characterize precisely under which conditions on the model of uncertainty used and on the loss function penalties robustification and regularization are equivalent.(b) We extend the characterization of robustification and regularization to matrix regression problems (matrix completion and Principal Component Analysis). 2021-10-27T20:29:06Z 2021-10-27T20:29:06Z 2018 2019-09-26T12:51:34Z Article http://purl.org/eprint/type/JournalArticle https://hdl.handle.net/1721.1/135747 en 10.1016/J.EJOR.2017.03.051 European Journal of Operational Research Creative Commons Attribution-NonCommercial-NoDerivs License http://creativecommons.org/licenses/by-nc-nd/4.0/ application/pdf Elsevier BV arXiv |
| spellingShingle | Bertsimas, Dimitris Copenhaver, Martin S Characterization of the equivalence of robustification and regularization in linear and matrix regression |
| title | Characterization of the equivalence of robustification and regularization in linear and matrix regression |
| title_full | Characterization of the equivalence of robustification and regularization in linear and matrix regression |
| title_fullStr | Characterization of the equivalence of robustification and regularization in linear and matrix regression |
| title_full_unstemmed | Characterization of the equivalence of robustification and regularization in linear and matrix regression |
| title_short | Characterization of the equivalence of robustification and regularization in linear and matrix regression |
| title_sort | characterization of the equivalence of robustification and regularization in linear and matrix regression |
| url | https://hdl.handle.net/1721.1/135747 |
| work_keys_str_mv | AT bertsimasdimitris characterizationoftheequivalenceofrobustificationandregularizationinlinearandmatrixregression AT copenhavermartins characterizationoftheequivalenceofrobustificationandregularizationinlinearandmatrixregression |