Bagging Regularizes
Intuitively, we expect that averaging --- or bagging --- different regressors with low correlation should smooth their behavior and be somewhat similar to regularization. In this note we make this intuition precise. Using an almost classical definition of stability, we prove that a certain form...
| Main Authors: | , , , |
|---|---|
| Language: | en_US |
| Published: |
2004
|
| Subjects: | |
| Online Access: | http://hdl.handle.net/1721.1/7268 |
| _version_ | 1826199337659531264 |
|---|---|
| author | Poggio, Tomaso Rifkin, Ryan Mukherjee, Sayan Rakhlin, Alex |
| author_facet | Poggio, Tomaso Rifkin, Ryan Mukherjee, Sayan Rakhlin, Alex |
| author_sort | Poggio, Tomaso |
| collection | MIT |
| description | Intuitively, we expect that averaging --- or bagging --- different regressors with low correlation should smooth their behavior and be somewhat similar to regularization. In this note we make this intuition precise. Using an almost classical definition of stability, we prove that a certain form of averaging provides generalization bounds with a rate of convergence of the same order as Tikhonov regularization --- similar to fashionable RKHS-based learning algorithms. |
| first_indexed | 2024-09-23T11:18:36Z |
| id | mit-1721.1/7268 |
| institution | Massachusetts Institute of Technology |
| language | en_US |
| last_indexed | 2024-09-23T11:18:36Z |
| publishDate | 2004 |
| record_format | dspace |
| spelling | mit-1721.1/72682019-04-15T00:40:26Z Bagging Regularizes Poggio, Tomaso Rifkin, Ryan Mukherjee, Sayan Rakhlin, Alex AI Bagging stability regularization Intuitively, we expect that averaging --- or bagging --- different regressors with low correlation should smooth their behavior and be somewhat similar to regularization. In this note we make this intuition precise. Using an almost classical definition of stability, we prove that a certain form of averaging provides generalization bounds with a rate of convergence of the same order as Tikhonov regularization --- similar to fashionable RKHS-based learning algorithms. 2004-10-20T21:04:57Z 2004-10-20T21:04:57Z 2002-03-01 AIM-2002-003 CBCL-214 http://hdl.handle.net/1721.1/7268 en_US AIM-2002-003 CBCL-214 7 p. 906324 bytes 285651 bytes application/postscript application/pdf application/postscript application/pdf |
| spellingShingle | AI Bagging stability regularization Poggio, Tomaso Rifkin, Ryan Mukherjee, Sayan Rakhlin, Alex Bagging Regularizes |
| title | Bagging Regularizes |
| title_full | Bagging Regularizes |
| title_fullStr | Bagging Regularizes |
| title_full_unstemmed | Bagging Regularizes |
| title_short | Bagging Regularizes |
| title_sort | bagging regularizes |
| topic | AI Bagging stability regularization |
| url | http://hdl.handle.net/1721.1/7268 |
| work_keys_str_mv | AT poggiotomaso baggingregularizes AT rifkinryan baggingregularizes AT mukherjeesayan baggingregularizes AT rakhlinalex baggingregularizes |