Asymptotics of Gaussian Regularized Least-Squares
We consider regularized least-squares (RLS) with a Gaussian kernel. Weprove that if we let the Gaussian bandwidth $\sigma \rightarrow\infty$ while letting the regularization parameter $\lambda\rightarrow 0$, the RLS solution tends to a polynomial whose order iscontrolled by the relative rates of dec...
| Main Authors: | , |
|---|---|
| Language: | en_US |
| Published: |
2005
|
| Subjects: | |
| Online Access: | http://hdl.handle.net/1721.1/30577 |
| _version_ | 1811079384023433216 |
|---|---|
| author | Lippert, Ross Rifkin, Ryan |
| author_facet | Lippert, Ross Rifkin, Ryan |
| author_sort | Lippert, Ross |
| collection | MIT |
| description | We consider regularized least-squares (RLS) with a Gaussian kernel. Weprove that if we let the Gaussian bandwidth $\sigma \rightarrow\infty$ while letting the regularization parameter $\lambda\rightarrow 0$, the RLS solution tends to a polynomial whose order iscontrolled by the relative rates of decay of $\frac{1}{\sigma^2}$ and$\lambda$: if $\lambda = \sigma^{-(2k+1)}$, then, as $\sigma \rightarrow\infty$, the RLS solution tends to the $k$th order polynomial withminimal empirical error. We illustrate the result with an example. |
| first_indexed | 2024-09-23T11:14:09Z |
| id | mit-1721.1/30577 |
| institution | Massachusetts Institute of Technology |
| language | en_US |
| last_indexed | 2024-09-23T11:14:09Z |
| publishDate | 2005 |
| record_format | dspace |
| spelling | mit-1721.1/305772019-04-12T13:39:30Z Asymptotics of Gaussian Regularized Least-Squares Lippert, Ross Rifkin, Ryan AI machine learning regularization We consider regularized least-squares (RLS) with a Gaussian kernel. Weprove that if we let the Gaussian bandwidth $\sigma \rightarrow\infty$ while letting the regularization parameter $\lambda\rightarrow 0$, the RLS solution tends to a polynomial whose order iscontrolled by the relative rates of decay of $\frac{1}{\sigma^2}$ and$\lambda$: if $\lambda = \sigma^{-(2k+1)}$, then, as $\sigma \rightarrow\infty$, the RLS solution tends to the $k$th order polynomial withminimal empirical error. We illustrate the result with an example. 2005-12-22T02:40:10Z 2005-12-22T02:40:10Z 2005-10-20 MIT-CSAIL-TR-2005-067 AIM-2005-030 CBCL-257 http://hdl.handle.net/1721.1/30577 en_US Massachusetts Institute of Technology Computer Science and Artificial Intelligence Laboratory 1 p. 7286963 bytes 527607 bytes application/postscript application/pdf application/postscript application/pdf |
| spellingShingle | AI machine learning regularization Lippert, Ross Rifkin, Ryan Asymptotics of Gaussian Regularized Least-Squares |
| title | Asymptotics of Gaussian Regularized Least-Squares |
| title_full | Asymptotics of Gaussian Regularized Least-Squares |
| title_fullStr | Asymptotics of Gaussian Regularized Least-Squares |
| title_full_unstemmed | Asymptotics of Gaussian Regularized Least-Squares |
| title_short | Asymptotics of Gaussian Regularized Least-Squares |
| title_sort | asymptotics of gaussian regularized least squares |
| topic | AI machine learning regularization |
| url | http://hdl.handle.net/1721.1/30577 |
| work_keys_str_mv | AT lippertross asymptoticsofgaussianregularizedleastsquares AT rifkinryan asymptoticsofgaussianregularizedleastsquares |