Easy-but-effective domain sub-similarity learning for transfer regression
Transfer covariance function, which can model domain similarity and adaptively control the knowledge transfer across domains, is widely used in transfer learning. In this paper, we concentrate on Gaussian process (GP) models using a transfer covariance function for regression problems in a black-box...
| Main Authors: | , , , |
|---|---|
| Other Authors: | |
| Format: | Journal Article |
| Language: | English |
| Published: |
2021
|
| Subjects: | |
| Online Access: | https://hdl.handle.net/10356/147569 |
| _version_ | 1826115478064463872 |
|---|---|
| author | Wei, Pengfei Sagarna, Ramon Ke, Yiping Ong, Yew-Soon |
| author2 | School of Computer Science and Engineering |
| author_facet | School of Computer Science and Engineering Wei, Pengfei Sagarna, Ramon Ke, Yiping Ong, Yew-Soon |
| author_sort | Wei, Pengfei |
| collection | NTU |
| description | Transfer covariance function, which can model domain similarity and adaptively control the knowledge transfer across domains, is widely used in transfer learning. In this paper, we concentrate on Gaussian process (GP) models using a transfer covariance function for regression problems in a black-box learning scenario. Precisely, we investigate a family of rather general transfer covariance functions, T*, that can model the heterogeneous sub-similarities of domains through multiple kernel learning. A necessary and sufficient condition to obtain valid GPs using T* (GPT* ) for any data is given. This condition becomes specially handy for practical applications as (i) it enables semantic interpretations of the sub-similarities and (ii) it can readily be used for model learning. In particular, we propose a computationally inexpensive model learning rule that can explicitly capture different sub-similarities of domains. We propose two instantiations of GPT* , one with a set of predefined constant base kernels and one with a set of learnable parametric base kernels. Extensive experiments on 36 synthetic transfer tasks and 12 real-world transfer tasks demonstrate the effectiveness of GPT* on the sub-similarity capture and the transfer performance. |
| first_indexed | 2024-10-01T03:55:49Z |
| format | Journal Article |
| id | ntu-10356/147569 |
| institution | Nanyang Technological University |
| language | English |
| last_indexed | 2024-10-01T03:55:49Z |
| publishDate | 2021 |
| record_format | dspace |
| spelling | ntu-10356/1475692022-01-03T08:28:34Z Easy-but-effective domain sub-similarity learning for transfer regression Wei, Pengfei Sagarna, Ramon Ke, Yiping Ong, Yew-Soon School of Computer Science and Engineering Engineering::Computer science and engineering::Computing methodologies::Artificial intelligence Transfer Regression Transfer Covariance Function Gaussian Process Transfer covariance function, which can model domain similarity and adaptively control the knowledge transfer across domains, is widely used in transfer learning. In this paper, we concentrate on Gaussian process (GP) models using a transfer covariance function for regression problems in a black-box learning scenario. Precisely, we investigate a family of rather general transfer covariance functions, T*, that can model the heterogeneous sub-similarities of domains through multiple kernel learning. A necessary and sufficient condition to obtain valid GPs using T* (GPT* ) for any data is given. This condition becomes specially handy for practical applications as (i) it enables semantic interpretations of the sub-similarities and (ii) it can readily be used for model learning. In particular, we propose a computationally inexpensive model learning rule that can explicitly capture different sub-similarities of domains. We propose two instantiations of GPT* , one with a set of predefined constant base kernels and one with a set of learnable parametric base kernels. Extensive experiments on 36 synthetic transfer tasks and 12 real-world transfer tasks demonstrate the effectiveness of GPT* on the sub-similarity capture and the transfer performance. Nanyang Technological University National Research Foundation (NRF) Accepted version This research is partially supported by the National Research Foundation (NRF), Singapore, under the Singapore Data Science Consortium (SDSC) International Research Collaboration Grant No. SDSC-2020-004. Any opinions, findings and conclusions or recommendations expressed in this material are those of the author(s) and do not reflect the views of NRF and SDSC. Yew- Soon Ong acknowledges the support of the Singtel Cognitive and Artificial Intelligence Lab for Enterprises (SCALE@NTU), which is a collaboration between Singapore Telecommunications Limited (Singtel) and Nanyang Technological University (NTU) that is funded by the Singapore Government through the Industry Alignment Fund - Industry Collaboration Projects Grant. 2021-04-06T06:38:36Z 2021-04-06T06:38:36Z 2020 Journal Article Wei, P., Sagarna, R., Ke, Y. & Ong, Y. (2020). Easy-but-effective domain sub-similarity learning for transfer regression. IEEE Transactions On Knowledge and Data Engineering. https://dx.doi.org/10.1109/TKDE.2020.3039806 1041-4347 https://hdl.handle.net/10356/147569 10.1109/TKDE.2020.3039806 en SDSC-2020-004 IEEE Transactions on Knowledge and Data Engineering © 2020 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works. The published version is available at: https://doi.org/10.1109/TKDE.2020.3039806. application/pdf |
| spellingShingle | Engineering::Computer science and engineering::Computing methodologies::Artificial intelligence Transfer Regression Transfer Covariance Function Gaussian Process Wei, Pengfei Sagarna, Ramon Ke, Yiping Ong, Yew-Soon Easy-but-effective domain sub-similarity learning for transfer regression |
| title | Easy-but-effective domain sub-similarity learning for transfer regression |
| title_full | Easy-but-effective domain sub-similarity learning for transfer regression |
| title_fullStr | Easy-but-effective domain sub-similarity learning for transfer regression |
| title_full_unstemmed | Easy-but-effective domain sub-similarity learning for transfer regression |
| title_short | Easy-but-effective domain sub-similarity learning for transfer regression |
| title_sort | easy but effective domain sub similarity learning for transfer regression |
| topic | Engineering::Computer science and engineering::Computing methodologies::Artificial intelligence Transfer Regression Transfer Covariance Function Gaussian Process |
| url | https://hdl.handle.net/10356/147569 |
| work_keys_str_mv | AT weipengfei easybuteffectivedomainsubsimilaritylearningfortransferregression AT sagarnaramon easybuteffectivedomainsubsimilaritylearningfortransferregression AT keyiping easybuteffectivedomainsubsimilaritylearningfortransferregression AT ongyewsoon easybuteffectivedomainsubsimilaritylearningfortransferregression |