Extrinsic Bayesian Optimization on Manifolds
We propose an extrinsic Bayesian optimization (eBO) framework for general optimization problems on manifolds. Bayesian optimization algorithms build a surrogate of the objective function by employing Gaussian processes and utilizing the uncertainty in that surrogate by deriving an acquisition functi...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-02-01
|
| Series: | Algorithms |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1999-4893/16/2/117 |
| _version_ | 1797622820771987456 |
|---|---|
| author | Yihao Fang Mu Niu Pokman Cheung Lizhen Lin |
| author_facet | Yihao Fang Mu Niu Pokman Cheung Lizhen Lin |
| author_sort | Yihao Fang |
| collection | DOAJ |
| description | We propose an extrinsic Bayesian optimization (eBO) framework for general optimization problems on manifolds. Bayesian optimization algorithms build a surrogate of the objective function by employing Gaussian processes and utilizing the uncertainty in that surrogate by deriving an acquisition function. This acquisition function represents the probability of improvement based on the kernel of the Gaussian process, which guides the search in the optimization process. The critical challenge for designing Bayesian optimization algorithms on manifolds lies in the difficulty of constructing valid covariance kernels for Gaussian processes on general manifolds. Our approach is to employ extrinsic Gaussian processes by first embedding the manifold onto some higher dimensional Euclidean space via equivariant embeddings and then constructing a valid covariance kernel on the image manifold after the embedding. This leads to efficient and scalable algorithms for optimization over complex manifolds. Simulation study and real data analyses are carried out to demonstrate the utilities of our eBO framework by applying the eBO to various optimization problems over manifolds such as the sphere, the Grassmannian, and the manifold of positive definite matrices. |
| first_indexed | 2024-03-11T09:15:40Z |
| format | Article |
| id | doaj.art-1902804263704984b6900ae8422c04a7 |
| institution | Directory Open Access Journal |
| issn | 1999-4893 |
| language | English |
| last_indexed | 2024-03-11T09:15:40Z |
| publishDate | 2023-02-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Algorithms |
| spelling | doaj.art-1902804263704984b6900ae8422c04a72023-11-16T18:38:00ZengMDPI AGAlgorithms1999-48932023-02-0116211710.3390/a16020117Extrinsic Bayesian Optimization on ManifoldsYihao Fang0Mu Niu1Pokman Cheung2Lizhen Lin3Department of Applied and Computational Mathematics and Statistics, University of Notre Dame, Notre Dame, IN 46556, USASchool of Mathematics and Statistics, University of Glasgow, Glasgow G12 8QQ, UKLondon, UKDepartment of Applied and Computational Mathematics and Statistics, University of Notre Dame, Notre Dame, IN 46556, USAWe propose an extrinsic Bayesian optimization (eBO) framework for general optimization problems on manifolds. Bayesian optimization algorithms build a surrogate of the objective function by employing Gaussian processes and utilizing the uncertainty in that surrogate by deriving an acquisition function. This acquisition function represents the probability of improvement based on the kernel of the Gaussian process, which guides the search in the optimization process. The critical challenge for designing Bayesian optimization algorithms on manifolds lies in the difficulty of constructing valid covariance kernels for Gaussian processes on general manifolds. Our approach is to employ extrinsic Gaussian processes by first embedding the manifold onto some higher dimensional Euclidean space via equivariant embeddings and then constructing a valid covariance kernel on the image manifold after the embedding. This leads to efficient and scalable algorithms for optimization over complex manifolds. Simulation study and real data analyses are carried out to demonstrate the utilities of our eBO framework by applying the eBO to various optimization problems over manifolds such as the sphere, the Grassmannian, and the manifold of positive definite matrices.https://www.mdpi.com/1999-4893/16/2/117Bayesian optimizationoptimizations on manifoldsembeddingextrinsic gaussian process |
| spellingShingle | Yihao Fang Mu Niu Pokman Cheung Lizhen Lin Extrinsic Bayesian Optimization on Manifolds Algorithms Bayesian optimization optimizations on manifolds embedding extrinsic gaussian process |
| title | Extrinsic Bayesian Optimization on Manifolds |
| title_full | Extrinsic Bayesian Optimization on Manifolds |
| title_fullStr | Extrinsic Bayesian Optimization on Manifolds |
| title_full_unstemmed | Extrinsic Bayesian Optimization on Manifolds |
| title_short | Extrinsic Bayesian Optimization on Manifolds |
| title_sort | extrinsic bayesian optimization on manifolds |
| topic | Bayesian optimization optimizations on manifolds embedding extrinsic gaussian process |
| url | https://www.mdpi.com/1999-4893/16/2/117 |
| work_keys_str_mv | AT yihaofang extrinsicbayesianoptimizationonmanifolds AT muniu extrinsicbayesianoptimizationonmanifolds AT pokmancheung extrinsicbayesianoptimizationonmanifolds AT lizhenlin extrinsicbayesianoptimizationonmanifolds |