Fast transfer Gaussian process regression with large-scale sources
In transfer learning, we aim to improve the predictive modeling of a target output by using the knowledge from some related source outputs. In real-world applications, the data from the target domain is often precious and hard to obtain, while the data from source domains is plentiful. Thus, since t...
| Main Authors: | , , , , |
|---|---|
| Other Authors: | |
| Format: | Journal Article |
| Language: | English |
| Published: |
2020
|
| Subjects: | |
| Online Access: | https://hdl.handle.net/10356/142637 |
| _version_ | 1826111162808270848 |
|---|---|
| author | Da, Bingshui Ong, Yew-Soon Gupta, Abhishek Feng, Liang Liu, Haitao |
| author2 | School of Computer Science and Engineering |
| author_facet | School of Computer Science and Engineering Da, Bingshui Ong, Yew-Soon Gupta, Abhishek Feng, Liang Liu, Haitao |
| author_sort | Da, Bingshui |
| collection | NTU |
| description | In transfer learning, we aim to improve the predictive modeling of a target output by using the knowledge from some related source outputs. In real-world applications, the data from the target domain is often precious and hard to obtain, while the data from source domains is plentiful. Thus, since the complexity of Gaussian process based multi-task/transfer learning approaches grows cubically with the total number of source＋ target observations, the method becomes increasingly impractical for large () source data inputs even with a small amount of target data. In order to scale known transfer Gaussian processes to large-scale source datasets, we propose an efficient aggregation model in this paper, which combines the predictions from distributed (small-scale) local experts in a principled manner. The proposed model inherits the advantages of single-task aggregation schemes, including efficient computation, analytically tractable inference, and straightforward parallelization during training and prediction. Further, a salient feature of the proposed method is the enhanced expressiveness in transfer learning — as a byproduct of flexible inter-task relationship modelings across different experts. When deploying such models in real-world applications, each local expert corresponds to a lightweight predictor that can be embedded in edge devices, thus catering to cases of online on-mote processing in fog computing settings. |
| first_indexed | 2024-10-01T02:46:09Z |
| format | Journal Article |
| id | ntu-10356/142637 |
| institution | Nanyang Technological University |
| language | English |
| last_indexed | 2024-10-01T02:46:09Z |
| publishDate | 2020 |
| record_format | dspace |
| spelling | ntu-10356/1426372020-06-26T01:46:07Z Fast transfer Gaussian process regression with large-scale sources Da, Bingshui Ong, Yew-Soon Gupta, Abhishek Feng, Liang Liu, Haitao School of Computer Science and Engineering Rolls-Royce@NTU Corporate Lab Engineering::Computer science and engineering Transfer Learning Large-scale In transfer learning, we aim to improve the predictive modeling of a target output by using the knowledge from some related source outputs. In real-world applications, the data from the target domain is often precious and hard to obtain, while the data from source domains is plentiful. Thus, since the complexity of Gaussian process based multi-task/transfer learning approaches grows cubically with the total number of source＋ target observations, the method becomes increasingly impractical for large () source data inputs even with a small amount of target data. In order to scale known transfer Gaussian processes to large-scale source datasets, we propose an efficient aggregation model in this paper, which combines the predictions from distributed (small-scale) local experts in a principled manner. The proposed model inherits the advantages of single-task aggregation schemes, including efficient computation, analytically tractable inference, and straightforward parallelization during training and prediction. Further, a salient feature of the proposed method is the enhanced expressiveness in transfer learning — as a byproduct of flexible inter-task relationship modelings across different experts. When deploying such models in real-world applications, each local expert corresponds to a lightweight predictor that can be embedded in edge devices, thus catering to cases of online on-mote processing in fog computing settings. NRF (Natl Research Foundation, S’pore) Accepted version 2020-06-26T01:46:07Z 2020-06-26T01:46:07Z 2018 Journal Article Da, B., Ong, Y.-S., Gupta, A., Feng, L., & Liu, H. (2019). Fast transfer Gaussian process regression with large-scale sources. Knowledge-Based Systems, 165, 208-218. doi:10.1016/j.knosys.2018.11.029 0950-7051 https://hdl.handle.net/10356/142637 10.1016/j.knosys.2018.11.029 2-s2.0-85059035742 165 208 218 en Knowledge-Based Systems © 2018 Elsevier B.V. All rights reserved. This paper was published in Knowledge-Based Systems and is made available with permission of Elsevier B.V. application/pdf |
| spellingShingle | Engineering::Computer science and engineering Transfer Learning Large-scale Da, Bingshui Ong, Yew-Soon Gupta, Abhishek Feng, Liang Liu, Haitao Fast transfer Gaussian process regression with large-scale sources |
| title | Fast transfer Gaussian process regression with large-scale sources |
| title_full | Fast transfer Gaussian process regression with large-scale sources |
| title_fullStr | Fast transfer Gaussian process regression with large-scale sources |
| title_full_unstemmed | Fast transfer Gaussian process regression with large-scale sources |
| title_short | Fast transfer Gaussian process regression with large-scale sources |
| title_sort | fast transfer gaussian process regression with large scale sources |
| topic | Engineering::Computer science and engineering Transfer Learning Large-scale |
| url | https://hdl.handle.net/10356/142637 |
| work_keys_str_mv | AT dabingshui fasttransfergaussianprocessregressionwithlargescalesources AT ongyewsoon fasttransfergaussianprocessregressionwithlargescalesources AT guptaabhishek fasttransfergaussianprocessregressionwithlargescalesources AT fengliang fasttransfergaussianprocessregressionwithlargescalesources AT liuhaitao fasttransfergaussianprocessregressionwithlargescalesources |