Adaptive Kernel Methods Using the Balancing Principle
The regularization parameter choice is a fundamental problem in supervised learning since the performance of most algorithms crucially depends on the choice of one or more of such parameters. In particular a main theoretical issue regards the amount of prior knowledge on the problem needed to suitab...
| Main Authors: | , , |
|---|---|
| Other Authors: | |
| Published: |
2008
|
| Subjects: | |
| Online Access: | http://hdl.handle.net/1721.1/42896 |
| _version_ | 1811097977423396864 |
|---|---|
| author | Rosasco, Lorenzo Pereverzyev, Sergei De Vito, Ernesto |
| author2 | Tomaso Poggio |
| author_facet | Tomaso Poggio Rosasco, Lorenzo Pereverzyev, Sergei De Vito, Ernesto |
| author_sort | Rosasco, Lorenzo |
| collection | MIT |
| description | The regularization parameter choice is a fundamental problem in supervised learning since the performance of most algorithms crucially depends on the choice of one or more of such parameters. In particular a main theoretical issue regards the amount of prior knowledge on the problem needed to suitably choose the regularization parameter and obtain learning rates. In this paper we present a strategy, the balancing principle, to choose the regularization parameter without knowledge of the regularity of the target function. Such a choice adaptively achieves the best error rate. Our main result applies to regularization algorithms in reproducing kernel Hilbert space with the square loss, though we also study how a similar principle can be used in other situations. As a straightforward corollary we can immediately derive adaptive parameter choice for various kernel methods recently studied. Numerical experiments with the proposed parameter choice rules are also presented. |
| first_indexed | 2024-09-23T17:08:00Z |
| id | mit-1721.1/42896 |
| institution | Massachusetts Institute of Technology |
| last_indexed | 2024-09-23T17:08:00Z |
| publishDate | 2008 |
| record_format | dspace |
| spelling | mit-1721.1/428962019-04-12T09:57:49Z Adaptive Kernel Methods Using the Balancing Principle Rosasco, Lorenzo Pereverzyev, Sergei De Vito, Ernesto Tomaso Poggio Center for Biological and Computational Learning (CBCL) Adaptive Model Selection Learning Theory Inverse Problems Regularization The regularization parameter choice is a fundamental problem in supervised learning since the performance of most algorithms crucially depends on the choice of one or more of such parameters. In particular a main theoretical issue regards the amount of prior knowledge on the problem needed to suitably choose the regularization parameter and obtain learning rates. In this paper we present a strategy, the balancing principle, to choose the regularization parameter without knowledge of the regularity of the target function. Such a choice adaptively achieves the best error rate. Our main result applies to regularization algorithms in reproducing kernel Hilbert space with the square loss, though we also study how a similar principle can be used in other situations. As a straightforward corollary we can immediately derive adaptive parameter choice for various kernel methods recently studied. Numerical experiments with the proposed parameter choice rules are also presented. 2008-10-17T15:30:10Z 2008-10-17T15:30:10Z 2008-10-16 http://hdl.handle.net/1721.1/42896 MIT-CSAIL-TR-2008-062 CBCL-275 24 p. application/pdf application/postscript |
| spellingShingle | Adaptive Model Selection Learning Theory Inverse Problems Regularization Rosasco, Lorenzo Pereverzyev, Sergei De Vito, Ernesto Adaptive Kernel Methods Using the Balancing Principle |
| title | Adaptive Kernel Methods Using the Balancing Principle |
| title_full | Adaptive Kernel Methods Using the Balancing Principle |
| title_fullStr | Adaptive Kernel Methods Using the Balancing Principle |
| title_full_unstemmed | Adaptive Kernel Methods Using the Balancing Principle |
| title_short | Adaptive Kernel Methods Using the Balancing Principle |
| title_sort | adaptive kernel methods using the balancing principle |
| topic | Adaptive Model Selection Learning Theory Inverse Problems Regularization |
| url | http://hdl.handle.net/1721.1/42896 |
| work_keys_str_mv | AT rosascolorenzo adaptivekernelmethodsusingthebalancingprinciple AT pereverzyevsergei adaptivekernelmethodsusingthebalancingprinciple AT devitoernesto adaptivekernelmethodsusingthebalancingprinciple |