On the Noise Model of Support Vector Machine Regression
Support Vector Machines Regression (SVMR) is a regression technique which has been recently introduced by V. Vapnik and his collaborators (Vapnik, 1995; Vapnik, Golowich and Smola, 1996). In SVMR the goodness of fit is measured not by the usual quadratic loss function (the mean square error),...
| Main Authors: | , , |
|---|---|
| Language: | en_US |
| Published: |
2004
|
| Online Access: | http://hdl.handle.net/1721.1/7259 |
| _version_ | 1826209280354680832 |
|---|---|
| author | Pontil, Massimiliano Mukherjee, Sayan Girosi, Federico |
| author_facet | Pontil, Massimiliano Mukherjee, Sayan Girosi, Federico |
| author_sort | Pontil, Massimiliano |
| collection | MIT |
| description | Support Vector Machines Regression (SVMR) is a regression technique which has been recently introduced by V. Vapnik and his collaborators (Vapnik, 1995; Vapnik, Golowich and Smola, 1996). In SVMR the goodness of fit is measured not by the usual quadratic loss function (the mean square error), but by a different loss function called Vapnik"s $epsilon$- insensitive loss function, which is similar to the "robust" loss functions introduced by Huber (Huber, 1981). The quadratic loss function is well justified under the assumption of Gaussian additive noise. However, the noise model underlying the choice of Vapnik's loss function is less clear. In this paper the use of Vapnik's loss function is shown to be equivalent to a model of additive and Gaussian noise, where the variance and mean of the Gaussian are random variables. The probability distributions for the variance and mean will be stated explicitly. While this work is presented in the framework of SVMR, it can be extended to justify non-quadratic loss functions in any Maximum Likelihood or Maximum A Posteriori approach. It applies not only to Vapnik's loss function, but to a much broader class of loss functions. |
| first_indexed | 2024-09-23T14:20:22Z |
| id | mit-1721.1/7259 |
| institution | Massachusetts Institute of Technology |
| language | en_US |
| last_indexed | 2024-09-23T14:20:22Z |
| publishDate | 2004 |
| record_format | dspace |
| spelling | mit-1721.1/72592019-04-15T00:40:23Z On the Noise Model of Support Vector Machine Regression Pontil, Massimiliano Mukherjee, Sayan Girosi, Federico Support Vector Machines Regression (SVMR) is a regression technique which has been recently introduced by V. Vapnik and his collaborators (Vapnik, 1995; Vapnik, Golowich and Smola, 1996). In SVMR the goodness of fit is measured not by the usual quadratic loss function (the mean square error), but by a different loss function called Vapnik"s $epsilon$- insensitive loss function, which is similar to the "robust" loss functions introduced by Huber (Huber, 1981). The quadratic loss function is well justified under the assumption of Gaussian additive noise. However, the noise model underlying the choice of Vapnik's loss function is less clear. In this paper the use of Vapnik's loss function is shown to be equivalent to a model of additive and Gaussian noise, where the variance and mean of the Gaussian are random variables. The probability distributions for the variance and mean will be stated explicitly. While this work is presented in the framework of SVMR, it can be extended to justify non-quadratic loss functions in any Maximum Likelihood or Maximum A Posteriori approach. It applies not only to Vapnik's loss function, but to a much broader class of loss functions. 2004-10-20T21:04:28Z 2004-10-20T21:04:28Z 1998-10-01 AIM-1651 CBCL-168 http://hdl.handle.net/1721.1/7259 en_US AIM-1651 CBCL-168 2520205 bytes 186978 bytes application/postscript application/pdf application/postscript application/pdf |
| spellingShingle | Pontil, Massimiliano Mukherjee, Sayan Girosi, Federico On the Noise Model of Support Vector Machine Regression |
| title | On the Noise Model of Support Vector Machine Regression |
| title_full | On the Noise Model of Support Vector Machine Regression |
| title_fullStr | On the Noise Model of Support Vector Machine Regression |
| title_full_unstemmed | On the Noise Model of Support Vector Machine Regression |
| title_short | On the Noise Model of Support Vector Machine Regression |
| title_sort | on the noise model of support vector machine regression |
| url | http://hdl.handle.net/1721.1/7259 |
| work_keys_str_mv | AT pontilmassimiliano onthenoisemodelofsupportvectormachineregression AT mukherjeesayan onthenoisemodelofsupportvectormachineregression AT girosifederico onthenoisemodelofsupportvectormachineregression |