Properties of Support Vector Machines
Support Vector Machines (SVMs) perform pattern recognition between two point classes by finding a decision surface determined by certain points of the training set, termed Support Vectors (SV). This surface, which in some feature space of possibly infinite dimension can be regarded as a hyperplane,...
| Päätekijät: | , |
|---|---|
| Kieli: | en_US |
| Julkaistu: |
2004
|
| Linkit: | http://hdl.handle.net/1721.1/7246 |
| _version_ | 1826188506500694016 |
|---|---|
| author | Pontil, Massimiliano Verri, Alessandro |
| author_facet | Pontil, Massimiliano Verri, Alessandro |
| author_sort | Pontil, Massimiliano |
| collection | MIT |
| description | Support Vector Machines (SVMs) perform pattern recognition between two point classes by finding a decision surface determined by certain points of the training set, termed Support Vectors (SV). This surface, which in some feature space of possibly infinite dimension can be regarded as a hyperplane, is obtained from the solution of a problem of quadratic programming that depends on a regularization parameter. In this paper we study some mathematical properties of support vectors and show that the decision surface can be written as the sum of two orthogonal terms, the first depending only on the margin vectors (which are SVs lying on the margin), the second proportional to the regularization parameter. For almost all values of the parameter, this enables us to predict how the decision surface varies for small parameter changes. In the special but important case of feature space of finite dimension m, we also show that there are at most m+1 margin vectors and observe that m+1 SVs are usually sufficient to fully determine the decision surface. For relatively small m this latter result leads to a consistent reduction of the SV number. |
| first_indexed | 2024-09-23T08:00:42Z |
| id | mit-1721.1/7246 |
| institution | Massachusetts Institute of Technology |
| language | en_US |
| last_indexed | 2024-09-23T08:00:42Z |
| publishDate | 2004 |
| record_format | dspace |
| spelling | mit-1721.1/72462019-04-09T16:05:35Z Properties of Support Vector Machines Pontil, Massimiliano Verri, Alessandro Support Vector Machines (SVMs) perform pattern recognition between two point classes by finding a decision surface determined by certain points of the training set, termed Support Vectors (SV). This surface, which in some feature space of possibly infinite dimension can be regarded as a hyperplane, is obtained from the solution of a problem of quadratic programming that depends on a regularization parameter. In this paper we study some mathematical properties of support vectors and show that the decision surface can be written as the sum of two orthogonal terms, the first depending only on the margin vectors (which are SVs lying on the margin), the second proportional to the regularization parameter. For almost all values of the parameter, this enables us to predict how the decision surface varies for small parameter changes. In the special but important case of feature space of finite dimension m, we also show that there are at most m+1 margin vectors and observe that m+1 SVs are usually sufficient to fully determine the decision surface. For relatively small m this latter result leads to a consistent reduction of the SV number. 2004-10-20T21:04:01Z 2004-10-20T21:04:01Z 1997-08-01 AIM-1612 CBCL-152 http://hdl.handle.net/1721.1/7246 en_US AIM-1612 CBCL-152 243488 bytes 406239 bytes application/postscript application/pdf application/postscript application/pdf |
| spellingShingle | Pontil, Massimiliano Verri, Alessandro Properties of Support Vector Machines |
| title | Properties of Support Vector Machines |
| title_full | Properties of Support Vector Machines |
| title_fullStr | Properties of Support Vector Machines |
| title_full_unstemmed | Properties of Support Vector Machines |
| title_short | Properties of Support Vector Machines |
| title_sort | properties of support vector machines |
| url | http://hdl.handle.net/1721.1/7246 |
| work_keys_str_mv | AT pontilmassimiliano propertiesofsupportvectormachines AT verrialessandro propertiesofsupportvectormachines |