Rectangularization of Gaussian process regression for optimization of hyperparameters
Gaussian process regression (GPR) is a powerful machine learning method which has recently enjoyed wider use, in particular in physical sciences. In its original formulation, GPR uses a square matrix of covariances among training data and can be viewed as linear regression problem with equal numbers...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Elsevier
2023-09-01
|
| Series: | Machine Learning with Applications |
| Subjects: | |
| Online Access: | http://www.sciencedirect.com/science/article/pii/S2666827023000403 |
| _version_ | 1827875700681474048 |
|---|---|
| author | Sergei Manzhos Manabu Ihara |
| author_facet | Sergei Manzhos Manabu Ihara |
| author_sort | Sergei Manzhos |
| collection | DOAJ |
| description | Gaussian process regression (GPR) is a powerful machine learning method which has recently enjoyed wider use, in particular in physical sciences. In its original formulation, GPR uses a square matrix of covariances among training data and can be viewed as linear regression problem with equal numbers of training data and basis functions. When data are sparse, avoidance of overfitting and optimization of hyperparameters of GPR are difficult, in particular in high-dimensional spaces where the data sparsity issue cannot practically be resolved by adding more data. Optimal choice of hyperparameters, however, determines success or failure of the application of the GPR method. We show that parameter optimization is facilitated by rectangularization of the defining equation of GPR. On the example of a 15-dimensional molecular potential energy surface we demonstrate that this approach allows effective hyperparameter tuning even with very sparse data. |
| first_indexed | 2024-03-12T17:11:42Z |
| format | Article |
| id | doaj.art-d01ad87bf2f44678a87f3eaae996b5f2 |
| institution | Directory Open Access Journal |
| issn | 2666-8270 |
| language | English |
| last_indexed | 2024-03-12T17:11:42Z |
| publishDate | 2023-09-01 |
| publisher | Elsevier |
| record_format | Article |
| series | Machine Learning with Applications |
| spelling | doaj.art-d01ad87bf2f44678a87f3eaae996b5f22023-08-06T04:38:29ZengElsevierMachine Learning with Applications2666-82702023-09-0113100487Rectangularization of Gaussian process regression for optimization of hyperparametersSergei Manzhos0Manabu Ihara1Corresponding authors.; School of Materials and Chemical Technology, Tokyo Institute of Technology, Ookayama 2-12-1, Meguro-ku, Tokyo 152-8552, JapanCorresponding authors.; School of Materials and Chemical Technology, Tokyo Institute of Technology, Ookayama 2-12-1, Meguro-ku, Tokyo 152-8552, JapanGaussian process regression (GPR) is a powerful machine learning method which has recently enjoyed wider use, in particular in physical sciences. In its original formulation, GPR uses a square matrix of covariances among training data and can be viewed as linear regression problem with equal numbers of training data and basis functions. When data are sparse, avoidance of overfitting and optimization of hyperparameters of GPR are difficult, in particular in high-dimensional spaces where the data sparsity issue cannot practically be resolved by adding more data. Optimal choice of hyperparameters, however, determines success or failure of the application of the GPR method. We show that parameter optimization is facilitated by rectangularization of the defining equation of GPR. On the example of a 15-dimensional molecular potential energy surface we demonstrate that this approach allows effective hyperparameter tuning even with very sparse data.http://www.sciencedirect.com/science/article/pii/S2666827023000403Gaussian process regressionHyperparameter optimizationRectangular matrix |
| spellingShingle | Sergei Manzhos Manabu Ihara Rectangularization of Gaussian process regression for optimization of hyperparameters Machine Learning with Applications Gaussian process regression Hyperparameter optimization Rectangular matrix |
| title | Rectangularization of Gaussian process regression for optimization of hyperparameters |
| title_full | Rectangularization of Gaussian process regression for optimization of hyperparameters |
| title_fullStr | Rectangularization of Gaussian process regression for optimization of hyperparameters |
| title_full_unstemmed | Rectangularization of Gaussian process regression for optimization of hyperparameters |
| title_short | Rectangularization of Gaussian process regression for optimization of hyperparameters |
| title_sort | rectangularization of gaussian process regression for optimization of hyperparameters |
| topic | Gaussian process regression Hyperparameter optimization Rectangular matrix |
| url | http://www.sciencedirect.com/science/article/pii/S2666827023000403 |
| work_keys_str_mv | AT sergeimanzhos rectangularizationofgaussianprocessregressionforoptimizationofhyperparameters AT manabuihara rectangularizationofgaussianprocessregressionforoptimizationofhyperparameters |