An Experimental Study in Adaptive Kernel Selection for Bayesian Optimization
Bayesian Optimization has been widely used along with Gaussian Processes for solving expensive-to-evaluate black-box optimization problems. Overall, this approach has shown good results, and particularly for parameter tuning of machine learning algorithms. Nonetheless, Bayesian Optimization has to b...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2019-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/8936460/ |
| _version_ | 1819295697671815168 |
|---|---|
| author | Ibai Roman Roberto Santana Alexander Mendiburu Jose A. Lozano |
| author_facet | Ibai Roman Roberto Santana Alexander Mendiburu Jose A. Lozano |
| author_sort | Ibai Roman |
| collection | DOAJ |
| description | Bayesian Optimization has been widely used along with Gaussian Processes for solving expensive-to-evaluate black-box optimization problems. Overall, this approach has shown good results, and particularly for parameter tuning of machine learning algorithms. Nonetheless, Bayesian Optimization has to be also configured to achieve the best possible performance, being the selection of the kernel function a crucial choice. This paper investigates the convenience of adaptively changing the kernel function during the optimization process, instead of fixing it a priori. Six adaptive kernel selection strategies are introduced and tested in well-known synthetic and real-world optimization problems. In order to provide a more complete evaluation of the proposed kernel selection variants, two major kernel parameter setting approaches have been tested. According to our results, apart from having the advantage of removing the selection of the kernel out of the equation, adaptive kernel selection criteria show a better performance than fixed-kernel approaches. |
| first_indexed | 2024-12-24T04:46:20Z |
| format | Article |
| id | doaj.art-e2c2e5f278a2485487cb348ea6d16ba4 |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-12-24T04:46:20Z |
| publishDate | 2019-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-e2c2e5f278a2485487cb348ea6d16ba42022-12-21T17:14:40ZengIEEEIEEE Access2169-35362019-01-01718429418430210.1109/ACCESS.2019.29604988936460An Experimental Study in Adaptive Kernel Selection for Bayesian OptimizationIbai Roman0https://orcid.org/0000-0003-3574-6681Roberto Santana1https://orcid.org/0000-0002-1005-8535Alexander Mendiburu2https://orcid.org/0000-0002-7271-1931Jose A. Lozano3https://orcid.org/0000-0002-4683-8111Intelligent Systems Group, University of the Basque Country UPV/EHU, Donostia, SpainIntelligent Systems Group, University of the Basque Country UPV/EHU, Donostia, SpainIntelligent Systems Group, University of the Basque Country UPV/EHU, Donostia, SpainIntelligent Systems Group, University of the Basque Country UPV/EHU, Donostia, SpainBayesian Optimization has been widely used along with Gaussian Processes for solving expensive-to-evaluate black-box optimization problems. Overall, this approach has shown good results, and particularly for parameter tuning of machine learning algorithms. Nonetheless, Bayesian Optimization has to be also configured to achieve the best possible performance, being the selection of the kernel function a crucial choice. This paper investigates the convenience of adaptively changing the kernel function during the optimization process, instead of fixing it a priori. Six adaptive kernel selection strategies are introduced and tested in well-known synthetic and real-world optimization problems. In order to provide a more complete evaluation of the proposed kernel selection variants, two major kernel parameter setting approaches have been tested. According to our results, apart from having the advantage of removing the selection of the kernel out of the equation, adaptive kernel selection criteria show a better performance than fixed-kernel approaches.https://ieeexplore.ieee.org/document/8936460/Adaptive kernel selectionBayesian optimizationGaussian processparameter tuning |
| spellingShingle | Ibai Roman Roberto Santana Alexander Mendiburu Jose A. Lozano An Experimental Study in Adaptive Kernel Selection for Bayesian Optimization IEEE Access Adaptive kernel selection Bayesian optimization Gaussian process parameter tuning |
| title | An Experimental Study in Adaptive Kernel Selection for Bayesian Optimization |
| title_full | An Experimental Study in Adaptive Kernel Selection for Bayesian Optimization |
| title_fullStr | An Experimental Study in Adaptive Kernel Selection for Bayesian Optimization |
| title_full_unstemmed | An Experimental Study in Adaptive Kernel Selection for Bayesian Optimization |
| title_short | An Experimental Study in Adaptive Kernel Selection for Bayesian Optimization |
| title_sort | experimental study in adaptive kernel selection for bayesian optimization |
| topic | Adaptive kernel selection Bayesian optimization Gaussian process parameter tuning |
| url | https://ieeexplore.ieee.org/document/8936460/ |
| work_keys_str_mv | AT ibairoman anexperimentalstudyinadaptivekernelselectionforbayesianoptimization AT robertosantana anexperimentalstudyinadaptivekernelselectionforbayesianoptimization AT alexandermendiburu anexperimentalstudyinadaptivekernelselectionforbayesianoptimization AT josealozano anexperimentalstudyinadaptivekernelselectionforbayesianoptimization AT ibairoman experimentalstudyinadaptivekernelselectionforbayesianoptimization AT robertosantana experimentalstudyinadaptivekernelselectionforbayesianoptimization AT alexandermendiburu experimentalstudyinadaptivekernelselectionforbayesianoptimization AT josealozano experimentalstudyinadaptivekernelselectionforbayesianoptimization |