For interpolating kernel machines, the minimum norm ERM solution is the most stable
We study the average CVloo stability of kernel ridge-less regression and derive corresponding risk bounds. We show that the interpolating solution with minimum norm has the best CVloo stability, which in turn is controlled by the condition number of the empirical kernel matrix. The latter can be cha...
| Main Authors: | , , |
|---|---|
| Format: | Technical Report |
| Published: |
Center for Brains, Minds and Machines (CBMM)
2020
|
| Online Access: | https://hdl.handle.net/1721.1/125927 |
| _version_ | 1826216503785029632 |
|---|---|
| author | Rangamani, Akshay Rosasco, Lorenzo Poggio, Tomaso |
| author_facet | Rangamani, Akshay Rosasco, Lorenzo Poggio, Tomaso |
| author_sort | Rangamani, Akshay |
| collection | MIT |
| description | We study the average CVloo stability of kernel ridge-less regression and derive corresponding risk bounds. We show that the interpolating solution with minimum norm has the best CVloo stability, which in turn is controlled by the condition number of the empirical kernel matrix. The latter can be characterized in the asymptotic regime where both the dimension and cardinality of the data go to infinity. Under the assumption of random kernel matrices, the corresponding test error follows a double descent curve. |
| first_indexed | 2024-09-23T16:48:24Z |
| format | Technical Report |
| id | mit-1721.1/125927 |
| institution | Massachusetts Institute of Technology |
| last_indexed | 2024-09-23T16:48:24Z |
| publishDate | 2020 |
| publisher | Center for Brains, Minds and Machines (CBMM) |
| record_format | dspace |
| spelling | mit-1721.1/1259272020-06-24T03:00:58Z For interpolating kernel machines, the minimum norm ERM solution is the most stable Rangamani, Akshay Rosasco, Lorenzo Poggio, Tomaso We study the average CVloo stability of kernel ridge-less regression and derive corresponding risk bounds. We show that the interpolating solution with minimum norm has the best CVloo stability, which in turn is controlled by the condition number of the empirical kernel matrix. The latter can be characterized in the asymptotic regime where both the dimension and cardinality of the data go to infinity. Under the assumption of random kernel matrices, the corresponding test error follows a double descent curve. This material is based upon work supported by the Center for Brains, Minds and Machines (CBMM), funded by NSF STC award CCF-1231216. 2020-06-23T13:18:04Z 2020-06-23T13:18:04Z 2020-06-22 Technical Report Working Paper Other https://hdl.handle.net/1721.1/125927 CBMM Memo;108 application/pdf Center for Brains, Minds and Machines (CBMM) |
| spellingShingle | Rangamani, Akshay Rosasco, Lorenzo Poggio, Tomaso For interpolating kernel machines, the minimum norm ERM solution is the most stable |
| title | For interpolating kernel machines, the minimum norm ERM solution is the most stable |
| title_full | For interpolating kernel machines, the minimum norm ERM solution is the most stable |
| title_fullStr | For interpolating kernel machines, the minimum norm ERM solution is the most stable |
| title_full_unstemmed | For interpolating kernel machines, the minimum norm ERM solution is the most stable |
| title_short | For interpolating kernel machines, the minimum norm ERM solution is the most stable |
| title_sort | for interpolating kernel machines the minimum norm erm solution is the most stable |
| url | https://hdl.handle.net/1721.1/125927 |
| work_keys_str_mv | AT rangamaniakshay forinterpolatingkernelmachinestheminimumnormermsolutionisthemoststable AT rosascolorenzo forinterpolatingkernelmachinestheminimumnormermsolutionisthemoststable AT poggiotomaso forinterpolatingkernelmachinestheminimumnormermsolutionisthemoststable |