Priors Stabilizers and Basis Functions: From Regularization to Radial, Tensor and Additive Splines
We had previously shown that regularization principles lead to approximation schemes, as Radial Basis Functions, which are equivalent to networks with one layer of hidden units, called Regularization Networks. In this paper we show that regularization networks encompass a much broader range of...
| Main Authors: | , , |
|---|---|
| Language: | en_US |
| Published: |
2004
|
| Subjects: | |
| Online Access: | http://hdl.handle.net/1721.1/7212 |
| _version_ | 1826201877227765760 |
|---|---|
| author | Girosi, Federico Jones, Michael Poggio, Tomaso |
| author_facet | Girosi, Federico Jones, Michael Poggio, Tomaso |
| author_sort | Girosi, Federico |
| collection | MIT |
| description | We had previously shown that regularization principles lead to approximation schemes, as Radial Basis Functions, which are equivalent to networks with one layer of hidden units, called Regularization Networks. In this paper we show that regularization networks encompass a much broader range of approximation schemes, including many of the popular general additive models, Breiman's hinge functions and some forms of Projection Pursuit Regression. In the probabilistic interpretation of regularization, the different classes of basis functions correspond to different classes of prior probabilities on the approximating function spaces, and therefore to different types of smoothness assumptions. In the final part of the paper, we also show a relation between activation functions of the Gaussian and sigmoidal type. |
| first_indexed | 2024-09-23T11:58:15Z |
| id | mit-1721.1/7212 |
| institution | Massachusetts Institute of Technology |
| language | en_US |
| last_indexed | 2024-09-23T11:58:15Z |
| publishDate | 2004 |
| record_format | dspace |
| spelling | mit-1721.1/72122019-04-10T09:58:44Z Priors Stabilizers and Basis Functions: From Regularization to Radial, Tensor and Additive Splines Girosi, Federico Jones, Michael Poggio, Tomaso regularization theory radial basis functions additivesmodels prior knowledge multilayer perceptrons We had previously shown that regularization principles lead to approximation schemes, as Radial Basis Functions, which are equivalent to networks with one layer of hidden units, called Regularization Networks. In this paper we show that regularization networks encompass a much broader range of approximation schemes, including many of the popular general additive models, Breiman's hinge functions and some forms of Projection Pursuit Regression. In the probabilistic interpretation of regularization, the different classes of basis functions correspond to different classes of prior probabilities on the approximating function spaces, and therefore to different types of smoothness assumptions. In the final part of the paper, we also show a relation between activation functions of the Gaussian and sigmoidal type. 2004-10-20T20:49:57Z 2004-10-20T20:49:57Z 1993-06-01 AIM-1430 CBCL-075 http://hdl.handle.net/1721.1/7212 en_US AIM-1430 CBCL-075 27 p. 768627 bytes 2437996 bytes application/octet-stream application/pdf application/octet-stream application/pdf |
| spellingShingle | regularization theory radial basis functions additivesmodels prior knowledge multilayer perceptrons Girosi, Federico Jones, Michael Poggio, Tomaso Priors Stabilizers and Basis Functions: From Regularization to Radial, Tensor and Additive Splines |
| title | Priors Stabilizers and Basis Functions: From Regularization to Radial, Tensor and Additive Splines |
| title_full | Priors Stabilizers and Basis Functions: From Regularization to Radial, Tensor and Additive Splines |
| title_fullStr | Priors Stabilizers and Basis Functions: From Regularization to Radial, Tensor and Additive Splines |
| title_full_unstemmed | Priors Stabilizers and Basis Functions: From Regularization to Radial, Tensor and Additive Splines |
| title_short | Priors Stabilizers and Basis Functions: From Regularization to Radial, Tensor and Additive Splines |
| title_sort | priors stabilizers and basis functions from regularization to radial tensor and additive splines |
| topic | regularization theory radial basis functions additivesmodels prior knowledge multilayer perceptrons |
| url | http://hdl.handle.net/1721.1/7212 |
| work_keys_str_mv | AT girosifederico priorsstabilizersandbasisfunctionsfromregularizationtoradialtensorandadditivesplines AT jonesmichael priorsstabilizersandbasisfunctionsfromregularizationtoradialtensorandadditivesplines AT poggiotomaso priorsstabilizersandbasisfunctionsfromregularizationtoradialtensorandadditivesplines |