Structure discovery in nonparametric regression through compositional kernel search
Despite its importance, choosing the structural form of the kernel in nonparametric regression remains a black art. We define a space of kernel structures which are built compositionally by adding and multiplying a small number of base kernels. We present a method for searching over this space of st...
| Main Authors: | , , , , |
|---|---|
| Other Authors: | |
| Format: | Article |
| Language: | en_US |
| Published: |
International Machine Learning Society
2015
|
| Online Access: | http://hdl.handle.net/1721.1/92896 https://orcid.org/0000-0002-1925-2035 |
| _version_ | 1826190233780092928 |
|---|---|
| author | Duvenaud, David Lloyd, James Robert Grosse, Roger Baker Tenenbaum, Joshua B. Ghahramani, Zoubin |
| author2 | Massachusetts Institute of Technology. Department of Brain and Cognitive Sciences |
| author_facet | Massachusetts Institute of Technology. Department of Brain and Cognitive Sciences Duvenaud, David Lloyd, James Robert Grosse, Roger Baker Tenenbaum, Joshua B. Ghahramani, Zoubin |
| author_sort | Duvenaud, David |
| collection | MIT |
| description | Despite its importance, choosing the structural form of the kernel in nonparametric regression remains a black art. We define a space of kernel structures which are built compositionally by adding and multiplying a small number of base kernels. We present a method for searching over this space of structures which mirrors the scientific discovery process. The learned structures can often decompose functions into interpretable components and enable long-range extrapolation on time-series datasets. Our structure search method outperforms many widely used kernels and kernel combination methods on a variety of prediction tasks. |
| first_indexed | 2024-09-23T08:37:07Z |
| format | Article |
| id | mit-1721.1/92896 |
| institution | Massachusetts Institute of Technology |
| language | en_US |
| last_indexed | 2024-09-23T08:37:07Z |
| publishDate | 2015 |
| publisher | International Machine Learning Society |
| record_format | dspace |
| spelling | mit-1721.1/928962022-09-30T09:59:34Z Structure discovery in nonparametric regression through compositional kernel search Duvenaud, David Lloyd, James Robert Grosse, Roger Baker Tenenbaum, Joshua B. Ghahramani, Zoubin Massachusetts Institute of Technology. Department of Brain and Cognitive Sciences Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science Grosse, Roger Baker Tenenbaum, Joshua B. Despite its importance, choosing the structural form of the kernel in nonparametric regression remains a black art. We define a space of kernel structures which are built compositionally by adding and multiplying a small number of base kernels. We present a method for searching over this space of structures which mirrors the scientific discovery process. The learned structures can often decompose functions into interpretable components and enable long-range extrapolation on time-series datasets. Our structure search method outperforms many widely used kernels and kernel combination methods on a variety of prediction tasks. Natural Sciences and Engineering Research Council of Canada Engineering and Physical Sciences Research Council (Grant EP/I036575/1) Google (Firm) 2015-01-15T19:04:48Z 2015-01-15T19:04:48Z 2013-06 Article http://purl.org/eprint/type/ConferencePaper http://hdl.handle.net/1721.1/92896 Duvenaud, David, James Robert Lloyd, Roger Grosse, Joshua B. Tenenbaum, and Zoubin Ghahramani. "Structure discovery in nonparametric regression through compositional kernel search." 30th International Conference on Machine Learning (June 2013). https://orcid.org/0000-0002-1925-2035 en_US Proceedings of the 30th International Conference on Machine Learning Creative Commons Attribution-Noncommercial-Share Alike http://creativecommons.org/licenses/by-nc-sa/4.0/ application/pdf International Machine Learning Society arXiv |
| spellingShingle | Duvenaud, David Lloyd, James Robert Grosse, Roger Baker Tenenbaum, Joshua B. Ghahramani, Zoubin Structure discovery in nonparametric regression through compositional kernel search |
| title | Structure discovery in nonparametric regression through compositional kernel search |
| title_full | Structure discovery in nonparametric regression through compositional kernel search |
| title_fullStr | Structure discovery in nonparametric regression through compositional kernel search |
| title_full_unstemmed | Structure discovery in nonparametric regression through compositional kernel search |
| title_short | Structure discovery in nonparametric regression through compositional kernel search |
| title_sort | structure discovery in nonparametric regression through compositional kernel search |
| url | http://hdl.handle.net/1721.1/92896 https://orcid.org/0000-0002-1925-2035 |
| work_keys_str_mv | AT duvenauddavid structurediscoveryinnonparametricregressionthroughcompositionalkernelsearch AT lloydjamesrobert structurediscoveryinnonparametricregressionthroughcompositionalkernelsearch AT grosserogerbaker structurediscoveryinnonparametricregressionthroughcompositionalkernelsearch AT tenenbaumjoshuab structurediscoveryinnonparametricregressionthroughcompositionalkernelsearch AT ghahramanizoubin structurediscoveryinnonparametricregressionthroughcompositionalkernelsearch |