On the existence of representer theorems in Banach spaces
We consider general regularisation and regularised interpolation problems for learning parameter vectors from data. In particular in Hilbert spaces regularisation methods have been applied very successfully, largely due to the well known representer theorem. Classical formulations of the theorem sta...
| Main Author: | |
|---|---|
| Other Authors: | |
| Format: | Thesis |
| Language: | English |
| Published: |
2019
|
| Subjects: |
| _version_ | 1817932951570087936 |
|---|---|
| author | Schlegel, K |
| author2 | Kristensen, J |
| author_facet | Kristensen, J Schlegel, K |
| author_sort | Schlegel, K |
| collection | OXFORD |
| description | We consider general regularisation and regularised interpolation problems for learning parameter vectors from data. In particular in Hilbert spaces regularisation methods have been applied very successfully, largely due to the well known representer theorem. Classical formulations of the theorem state that under certain conditions on the regulariser there exists a solution of the optimisation problem which is contained in a linear subspace spanned by the data points. This is at the core of kernel methods in machine learning as it significantly reduces the dimensionality and thus makes the problem computationally tractable. Most of the literature only deals with sufficient conditions on the regulariser for a representer theorem to hold, mostly in Hilbert spaces with some generalisations to certain classes of Banach spaces. In this work we give an essentially complete answer to the question of existence of representer theorems in general Banach spaces. This question had previously been answered for Hilbert spaces with an intuitive characterisation for differentiable regularisers. We show how the necessary and sufficient conditions extend to arbitrary Banach spaces and give the more intuitive geometric characterisation for a variety of classes of Banach spaces, which contain all spaces we know of which are commonly used in applications. We conjecture that the same characterisation can also be given for any other Banach space not currently covered by those classes. We further show that, if the learning relies on the linear representer theorem, in most cases the solution is actually independent of the regulariser and determined by the function space alone. This is interesting for two reasons. Firstly it means one is free to choose whichever regulariser is most suitable for the application at hand, whether this is computational efficiency or ease of calculations. Moreover it shows the importance of extending classical elements of learning theory such as kernel methods from Hilbert spaces to Banach spaces. |
| first_indexed | 2024-03-06T21:30:02Z |
| format | Thesis |
| id | oxford-uuid:44617eb0-1747-45ff-9e88-805dccc2738e |
| institution | University of Oxford |
| language | English |
| last_indexed | 2024-12-09T03:46:04Z |
| publishDate | 2019 |
| record_format | dspace |
| spelling | oxford-uuid:44617eb0-1747-45ff-9e88-805dccc2738e2024-12-07T18:02:03ZOn the existence of representer theorems in Banach spacesThesishttp://purl.org/coar/resource_type/c_db06uuid:44617eb0-1747-45ff-9e88-805dccc2738eBanach spacesLearning theoryEnglishHyrax Deposit2019Schlegel, KKristensen, JWe consider general regularisation and regularised interpolation problems for learning parameter vectors from data. In particular in Hilbert spaces regularisation methods have been applied very successfully, largely due to the well known representer theorem. Classical formulations of the theorem state that under certain conditions on the regulariser there exists a solution of the optimisation problem which is contained in a linear subspace spanned by the data points. This is at the core of kernel methods in machine learning as it significantly reduces the dimensionality and thus makes the problem computationally tractable. Most of the literature only deals with sufficient conditions on the regulariser for a representer theorem to hold, mostly in Hilbert spaces with some generalisations to certain classes of Banach spaces. In this work we give an essentially complete answer to the question of existence of representer theorems in general Banach spaces. This question had previously been answered for Hilbert spaces with an intuitive characterisation for differentiable regularisers. We show how the necessary and sufficient conditions extend to arbitrary Banach spaces and give the more intuitive geometric characterisation for a variety of classes of Banach spaces, which contain all spaces we know of which are commonly used in applications. We conjecture that the same characterisation can also be given for any other Banach space not currently covered by those classes. We further show that, if the learning relies on the linear representer theorem, in most cases the solution is actually independent of the regulariser and determined by the function space alone. This is interesting for two reasons. Firstly it means one is free to choose whichever regulariser is most suitable for the application at hand, whether this is computational efficiency or ease of calculations. Moreover it shows the importance of extending classical elements of learning theory such as kernel methods from Hilbert spaces to Banach spaces. |
| spellingShingle | Banach spaces Learning theory Schlegel, K On the existence of representer theorems in Banach spaces |
| title | On the existence of representer theorems in Banach spaces |
| title_full | On the existence of representer theorems in Banach spaces |
| title_fullStr | On the existence of representer theorems in Banach spaces |
| title_full_unstemmed | On the existence of representer theorems in Banach spaces |
| title_short | On the existence of representer theorems in Banach spaces |
| title_sort | on the existence of representer theorems in banach spaces |
| topic | Banach spaces Learning theory |
| work_keys_str_mv | AT schlegelk ontheexistenceofrepresentertheoremsinbanachspaces |