Tight bounds for the expected risk of linear classifiers and PAC-bayes finite-sample guarantees
We analyze the expected risk of linear classifiers for a fixed weight vector in the “minimax” setting. That is, we analyze the worst-case risk among all data distributions with a given mean and covariance. We provide a simpler proof of the tight polynomial-tail bound for general random variables. Fo...
| Main Authors: | , |
|---|---|
| Other Authors: | |
| Format: | Article |
| Language: | en_US |
| Published: |
Journal of Machine Learning Research
2018
|
| Online Access: | http://hdl.handle.net/1721.1/113045 https://orcid.org/0000-0003-0238-6384 https://orcid.org/0000-0002-2199-0379 |
| _version_ | 1811077661811802112 |
|---|---|
| author | Honorio Carrillo, Jean Jaakkola, Tommi S. |
| author2 | Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory |
| author_facet | Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory Honorio Carrillo, Jean Jaakkola, Tommi S. |
| author_sort | Honorio Carrillo, Jean |
| collection | MIT |
| description | We analyze the expected risk of linear classifiers for a fixed weight vector in the “minimax” setting. That is, we analyze the worst-case risk among all data distributions with a given mean and covariance. We provide a simpler proof of the tight polynomial-tail bound for general random variables. For sub-Gaussian random variables, we derive a novel tight exponential bound. We also provide new PAC-Bayes finite-sample guarantees when training data
is available. Our “minimax” generalization bounds are dimensionality-independent and O(√1/m) for m samples. |
| first_indexed | 2024-09-23T10:46:32Z |
| format | Article |
| id | mit-1721.1/113045 |
| institution | Massachusetts Institute of Technology |
| language | en_US |
| last_indexed | 2024-09-23T10:46:32Z |
| publishDate | 2018 |
| publisher | Journal of Machine Learning Research |
| record_format | dspace |
| spelling | mit-1721.1/1130452022-09-27T14:55:53Z Tight bounds for the expected risk of linear classifiers and PAC-bayes finite-sample guarantees Honorio Carrillo, Jean Jaakkola, Tommi S. Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science Honorio Carrillo, Jean Jaakkola, Tommi S. We analyze the expected risk of linear classifiers for a fixed weight vector in the “minimax” setting. That is, we analyze the worst-case risk among all data distributions with a given mean and covariance. We provide a simpler proof of the tight polynomial-tail bound for general random variables. For sub-Gaussian random variables, we derive a novel tight exponential bound. We also provide new PAC-Bayes finite-sample guarantees when training data is available. Our “minimax” generalization bounds are dimensionality-independent and O(√1/m) for m samples. 2018-01-10T16:52:14Z 2018-01-10T16:52:14Z 2014-04 Article http://purl.org/eprint/type/ConferencePaper 1938-7228 http://hdl.handle.net/1721.1/113045 Honorio, Jean and Tomi Jaakkola. "Tight bounds for the expected risk of linear classifiers and PAC-bayes finite-sample guarantees." Proceedings of the 17th International Conference on Artificial Intelligence and Statistics (AISTATS), 22-25 April 2014, Reykjavik, Iceland, Journal of Machine Learning Research, 2014. https://orcid.org/0000-0003-0238-6384 https://orcid.org/0000-0002-2199-0379 en_US http://proceedings.mlr.press/v33/ Proceedings of the 17th International Conference on Artificial Intelligence and Statistics (AISTATS) Creative Commons Attribution-Noncommercial-Share Alike http://creativecommons.org/licenses/by-nc-sa/4.0/ application/pdf Journal of Machine Learning Research MIT Web Domain |
| spellingShingle | Honorio Carrillo, Jean Jaakkola, Tommi S. Tight bounds for the expected risk of linear classifiers and PAC-bayes finite-sample guarantees |
| title | Tight bounds for the expected risk of linear classifiers and PAC-bayes finite-sample guarantees |
| title_full | Tight bounds for the expected risk of linear classifiers and PAC-bayes finite-sample guarantees |
| title_fullStr | Tight bounds for the expected risk of linear classifiers and PAC-bayes finite-sample guarantees |
| title_full_unstemmed | Tight bounds for the expected risk of linear classifiers and PAC-bayes finite-sample guarantees |
| title_short | Tight bounds for the expected risk of linear classifiers and PAC-bayes finite-sample guarantees |
| title_sort | tight bounds for the expected risk of linear classifiers and pac bayes finite sample guarantees |
| url | http://hdl.handle.net/1721.1/113045 https://orcid.org/0000-0003-0238-6384 https://orcid.org/0000-0002-2199-0379 |
| work_keys_str_mv | AT honoriocarrillojean tightboundsfortheexpectedriskoflinearclassifiersandpacbayesfinitesampleguarantees AT jaakkolatommis tightboundsfortheexpectedriskoflinearclassifiersandpacbayesfinitesampleguarantees |