Parallel Gaussian Process Regression with Low-Rank Covariance Matrix Approximations
Gaussian processes (GP) are Bayesian non- parametric models that are widely used for probabilistic regression. Unfortunately, it cannot scale well with large data nor perform real-time predictions due to its cubic time cost in the data size. This paper presents two parallel GP regression methods tha...
| Main Authors: | , , , , , |
|---|---|
| Other Authors: | |
| Format: | Article |
| Language: | en_US |
| Published: |
Association for Uncertainty in Artificial Intelligence Press
2014
|
| Online Access: | http://hdl.handle.net/1721.1/87022 https://orcid.org/0000-0002-8585-6566 |
| _version_ | 1811090774426648576 |
|---|---|
| author | Chen, Jie Cao, Nannan Low, Kian Hsiang Ouyang, Ruofei Colin Keng-Yan, Tan Jaillet, Patrick |
| author2 | Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science |
| author_facet | Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science Chen, Jie Cao, Nannan Low, Kian Hsiang Ouyang, Ruofei Colin Keng-Yan, Tan Jaillet, Patrick |
| author_sort | Chen, Jie |
| collection | MIT |
| description | Gaussian processes (GP) are Bayesian non- parametric models that are widely used for probabilistic regression. Unfortunately, it cannot scale well with large data nor perform real-time predictions due to its cubic time cost in the data size. This paper presents two parallel GP regression methods that exploit low-rank covariance matrix approximations for distributing the computational load among parallel machines to achieve time efficiency and scalability. We theoretically guarantee the predictive performance of our proposed parallel GPs to be equivalent to that of some centralized approximate GP regression methods: The computation of their centralized counterparts can be distributed among parallel machines, hence achieving greater time efficiency and scalability. We analytically compare the properties of our parallel GPs such as time, space, and communication complexity. Empirical evaluation on two real-world datasets in a cluster of 20 computing nodes shows that our parallel GPs are significantly more time-efficient and scalable than their centralized counterparts and exact/full GP while achieving predictive performances comparable to full GP. |
| first_indexed | 2024-09-23T14:51:45Z |
| format | Article |
| id | mit-1721.1/87022 |
| institution | Massachusetts Institute of Technology |
| language | en_US |
| last_indexed | 2024-09-23T14:51:45Z |
| publishDate | 2014 |
| publisher | Association for Uncertainty in Artificial Intelligence Press |
| record_format | dspace |
| spelling | mit-1721.1/870222022-10-01T23:00:13Z Parallel Gaussian Process Regression with Low-Rank Covariance Matrix Approximations Chen, Jie Cao, Nannan Low, Kian Hsiang Ouyang, Ruofei Colin Keng-Yan, Tan Jaillet, Patrick Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science Jaillet, Patrick Gaussian processes (GP) are Bayesian non- parametric models that are widely used for probabilistic regression. Unfortunately, it cannot scale well with large data nor perform real-time predictions due to its cubic time cost in the data size. This paper presents two parallel GP regression methods that exploit low-rank covariance matrix approximations for distributing the computational load among parallel machines to achieve time efficiency and scalability. We theoretically guarantee the predictive performance of our proposed parallel GPs to be equivalent to that of some centralized approximate GP regression methods: The computation of their centralized counterparts can be distributed among parallel machines, hence achieving greater time efficiency and scalability. We analytically compare the properties of our parallel GPs such as time, space, and communication complexity. Empirical evaluation on two real-world datasets in a cluster of 20 computing nodes shows that our parallel GPs are significantly more time-efficient and scalable than their centralized counterparts and exact/full GP while achieving predictive performances comparable to full GP. 2014-05-16T14:13:51Z 2014-05-16T14:13:51Z 2013-07 Article http://purl.org/eprint/type/ConferencePaper 1525-3384 http://hdl.handle.net/1721.1/87022 Chen, Jie, Nannan Cao, Kian Hsiang Low, Ruofei Ouyang, Colin Keng-Yan Tan, and Patrick Jaillet. "Parallel Gaussian Process Regression with Low-Rank Covariance Matrix Approximations." in Conference on Uncertainty in artificial Intelligence, Bellevue, Wash., USA. July 11-15, 2013. Edited by Ann Nicholson and Padhraic Smyth. (2013). pp.152-162. https://orcid.org/0000-0002-8585-6566 en_US http://www.auai.org/uai2013/prints/proceedings.pdf Proceedings of the Twenty-Ninth Conference on Uncertainty in Artificial Intelligence (UAI 2013) Creative Commons Attribution-Noncommercial-Share Alike http://creativecommons.org/licenses/by-nc-sa/4.0/ application/pdf Association for Uncertainty in Artificial Intelligence Press MIT web domain |
| spellingShingle | Chen, Jie Cao, Nannan Low, Kian Hsiang Ouyang, Ruofei Colin Keng-Yan, Tan Jaillet, Patrick Parallel Gaussian Process Regression with Low-Rank Covariance Matrix Approximations |
| title | Parallel Gaussian Process Regression with Low-Rank Covariance Matrix Approximations |
| title_full | Parallel Gaussian Process Regression with Low-Rank Covariance Matrix Approximations |
| title_fullStr | Parallel Gaussian Process Regression with Low-Rank Covariance Matrix Approximations |
| title_full_unstemmed | Parallel Gaussian Process Regression with Low-Rank Covariance Matrix Approximations |
| title_short | Parallel Gaussian Process Regression with Low-Rank Covariance Matrix Approximations |
| title_sort | parallel gaussian process regression with low rank covariance matrix approximations |
| url | http://hdl.handle.net/1721.1/87022 https://orcid.org/0000-0002-8585-6566 |
| work_keys_str_mv | AT chenjie parallelgaussianprocessregressionwithlowrankcovariancematrixapproximations AT caonannan parallelgaussianprocessregressionwithlowrankcovariancematrixapproximations AT lowkianhsiang parallelgaussianprocessregressionwithlowrankcovariancematrixapproximations AT ouyangruofei parallelgaussianprocessregressionwithlowrankcovariancematrixapproximations AT colinkengyantan parallelgaussianprocessregressionwithlowrankcovariancematrixapproximations AT jailletpatrick parallelgaussianprocessregressionwithlowrankcovariancematrixapproximations |