Locally linear support vector machines
Linear support vector machines (SVMs) have become popular for solving classification tasks due to their fast and simple online application to large scale data sets. However, many problems are not linearly separable. For these problems kernel-based SVMs are often used, but unlike their linear variant...
| Главные авторы: | , |
|---|---|
| Формат: | Conference item |
| Язык: | English |
| Опубликовано: |
Association for Computing Machinery
2011
|
| _version_ | 1826314895702884352 |
|---|---|
| author | Ladický, L Torr, PHS |
| author_facet | Ladický, L Torr, PHS |
| author_sort | Ladický, L |
| collection | OXFORD |
| description | Linear support vector machines (SVMs) have become popular for solving classification tasks due to their fast and simple online application to large scale data sets. However, many problems are not linearly separable. For these problems kernel-based SVMs are often used, but unlike their linear variant they suffer from various drawbacks in terms of computational and memory efficiency. Their response can be represented only as a function of the set of support vectors, which has been experimentally shown to grow linearly with the size of the training set. In this paper we propose a novel locally linear SVM classifier with smooth decision boundary and bounded curvature. We show how the functions defining the classifier can be approximated using local codings and show how this model can be optimized in an online fashion by performing stochastic gradient descent with the same convergence guarantees as standard gradient descent method for linear SVM. Our method achieves comparable performance to the state-of-the-art whilst being significantly faster than competing kernel SVMs. We generalise this model to locally finite dimensional kernel SVM. |
| first_indexed | 2024-12-09T03:15:57Z |
| format | Conference item |
| id | oxford-uuid:32cd9bfa-510d-4bfe-88d3-22cf775f4abc |
| institution | University of Oxford |
| language | English |
| last_indexed | 2024-12-09T03:15:57Z |
| publishDate | 2011 |
| publisher | Association for Computing Machinery |
| record_format | dspace |
| spelling | oxford-uuid:32cd9bfa-510d-4bfe-88d3-22cf775f4abc2024-10-22T10:25:03ZLocally linear support vector machinesConference itemhttp://purl.org/coar/resource_type/c_5794uuid:32cd9bfa-510d-4bfe-88d3-22cf775f4abcEnglishSymplectic ElementsAssociation for Computing Machinery2011Ladický, LTorr, PHSLinear support vector machines (SVMs) have become popular for solving classification tasks due to their fast and simple online application to large scale data sets. However, many problems are not linearly separable. For these problems kernel-based SVMs are often used, but unlike their linear variant they suffer from various drawbacks in terms of computational and memory efficiency. Their response can be represented only as a function of the set of support vectors, which has been experimentally shown to grow linearly with the size of the training set. In this paper we propose a novel locally linear SVM classifier with smooth decision boundary and bounded curvature. We show how the functions defining the classifier can be approximated using local codings and show how this model can be optimized in an online fashion by performing stochastic gradient descent with the same convergence guarantees as standard gradient descent method for linear SVM. Our method achieves comparable performance to the state-of-the-art whilst being significantly faster than competing kernel SVMs. We generalise this model to locally finite dimensional kernel SVM. |
| spellingShingle | Ladický, L Torr, PHS Locally linear support vector machines |
| title | Locally linear support vector machines |
| title_full | Locally linear support vector machines |
| title_fullStr | Locally linear support vector machines |
| title_full_unstemmed | Locally linear support vector machines |
| title_short | Locally linear support vector machines |
| title_sort | locally linear support vector machines |
| work_keys_str_mv | AT ladickyl locallylinearsupportvectormachines AT torrphs locallylinearsupportvectormachines |