Sufficient Conditions for Uniform Stability of Regularization Algorithms
In this paper, we study the stability and generalization properties of penalized empirical-risk minimization algorithms. We propose a set of properties of the penalty term that is sufficient to ensure uniform ?-stability: we show that if the penalty function satisfies a suitable convexity property,...
| Main Authors: | , , |
|---|---|
| Other Authors: | |
| Published: |
2009
|
| Subjects: | |
| Online Access: | http://hdl.handle.net/1721.1/49868 |
| _version_ | 1826198037104427008 |
|---|---|
| author | Poggio, Tomaso Rosasco, Lorenzo Wibisono, Andre |
| author2 | Tomaso Poggio |
| author_facet | Tomaso Poggio Poggio, Tomaso Rosasco, Lorenzo Wibisono, Andre |
| author_sort | Poggio, Tomaso |
| collection | MIT |
| description | In this paper, we study the stability and generalization properties of penalized empirical-risk minimization algorithms. We propose a set of properties of the penalty term that is sufficient to ensure uniform ?-stability: we show that if the penalty function satisfies a suitable convexity property, then the induced regularization algorithm is uniformly ?-stable. In particular, our results imply that regularization algorithms with penalty functions which are strongly convex on bounded domains are ?-stable. In view of the results in [3], uniform stability implies generalization, and moreover, consistency results can be easily obtained. We apply our results to show that â p regularization for 1 < p <= 2 and elastic-net regularization are uniformly ?-stable, and therefore generalize. |
| first_indexed | 2024-09-23T10:57:52Z |
| id | mit-1721.1/49868 |
| institution | Massachusetts Institute of Technology |
| last_indexed | 2024-09-23T10:57:52Z |
| publishDate | 2009 |
| record_format | dspace |
| spelling | mit-1721.1/498682019-04-10T11:27:19Z Sufficient Conditions for Uniform Stability of Regularization Algorithms Poggio, Tomaso Rosasco, Lorenzo Wibisono, Andre Tomaso Poggio Center for Biological and Computational Learning (CBCL) artificial intelligence theory computation learning In this paper, we study the stability and generalization properties of penalized empirical-risk minimization algorithms. We propose a set of properties of the penalty term that is sufficient to ensure uniform ?-stability: we show that if the penalty function satisfies a suitable convexity property, then the induced regularization algorithm is uniformly ?-stable. In particular, our results imply that regularization algorithms with penalty functions which are strongly convex on bounded domains are ?-stable. In view of the results in [3], uniform stability implies generalization, and moreover, consistency results can be easily obtained. We apply our results to show that â p regularization for 1 < p <= 2 and elastic-net regularization are uniformly ?-stable, and therefore generalize. 2009-12-01T21:15:05Z 2009-12-01T21:15:05Z 2009-12-01 http://hdl.handle.net/1721.1/49868 CBCL-284 MIT-CSAIL-TR-2009-060 16 p. application/pdf |
| spellingShingle | artificial intelligence theory computation learning Poggio, Tomaso Rosasco, Lorenzo Wibisono, Andre Sufficient Conditions for Uniform Stability of Regularization Algorithms |
| title | Sufficient Conditions for Uniform Stability of Regularization Algorithms |
| title_full | Sufficient Conditions for Uniform Stability of Regularization Algorithms |
| title_fullStr | Sufficient Conditions for Uniform Stability of Regularization Algorithms |
| title_full_unstemmed | Sufficient Conditions for Uniform Stability of Regularization Algorithms |
| title_short | Sufficient Conditions for Uniform Stability of Regularization Algorithms |
| title_sort | sufficient conditions for uniform stability of regularization algorithms |
| topic | artificial intelligence theory computation learning |
| url | http://hdl.handle.net/1721.1/49868 |
| work_keys_str_mv | AT poggiotomaso sufficientconditionsforuniformstabilityofregularizationalgorithms AT rosascolorenzo sufficientconditionsforuniformstabilityofregularizationalgorithms AT wibisonoandre sufficientconditionsforuniformstabilityofregularizationalgorithms |