Neural latent geometry search: product manifold inference via Gromov-Hausdorff-informed Bayesian optimization
Recent research indicates that the performance of machine learning models can be improved by aligning the geometry of the latent space with the underlying data structure. Rather than relying solely on Euclidean space, researchers have proposed using hyperbolic and spherical spaces with constant curv...
| Main Authors: | , , , , |
|---|---|
| Format: | Conference item |
| Jezik: | English |
| Izdano: |
Neural Information Processing Systems Foundation
2024
|
| _version_ | 1826313673033908224 |
|---|---|
| author | Sáez de Ocáriz Borde, H Arroyo, Á Morales López, I Posner, I Dong, X |
| author_facet | Sáez de Ocáriz Borde, H Arroyo, Á Morales López, I Posner, I Dong, X |
| author_sort | Sáez de Ocáriz Borde, H |
| collection | OXFORD |
| description | Recent research indicates that the performance of machine learning models can be improved by aligning the geometry of the latent space with the underlying data structure. Rather than relying solely on Euclidean space, researchers have proposed using hyperbolic and spherical spaces with constant curvature, or combinations thereof, to better model the latent space and enhance model performance. However, little attention has been given to the problem of automatically identifying the optimal latent geometry for the downstream task. We mathematically define this novel formulation and coin it as neural latent geometry search (NLGS). More specifically, we introduce an initial attempt to search for a latent geometry composed of a product of constant curvature model spaces with a small number of query evaluations, under some simplifying assumptions. To accomplish this, we propose a novel notion of distance between candidate latent geometries based on the Gromov-Hausdorff distance from metric geometry. In order to compute the Gromov-Hausdorff distance, we introduce a mapping function that enables the comparison of different manifolds by embedding them in a common high-dimensional ambient space. We then design a graph search space based on the notion of smoothness between latent geometries and employ the calculated distances as an additional inductive bias. Finally, we use Bayesian optimization to search for the optimal latent geometry in a query-efficient manner. This is a general method which can be applied to search for the optimal latent geometry for a variety of models and downstream tasks. We perform experiments on synthetic and real-world datasets to identify the optimal latent geometry for multiple machine learning problems. |
| first_indexed | 2024-09-25T04:18:43Z |
| format | Conference item |
| id | oxford-uuid:4e73dead-fa72-41c6-bae1-af7139a803ca |
| institution | University of Oxford |
| language | English |
| last_indexed | 2024-09-25T04:18:43Z |
| publishDate | 2024 |
| publisher | Neural Information Processing Systems Foundation |
| record_format | dspace |
| spelling | oxford-uuid:4e73dead-fa72-41c6-bae1-af7139a803ca2024-07-30T15:00:43ZNeural latent geometry search: product manifold inference via Gromov-Hausdorff-informed Bayesian optimization Conference itemhttp://purl.org/coar/resource_type/c_5794uuid:4e73dead-fa72-41c6-bae1-af7139a803caEnglishSymplectic ElementsNeural Information Processing Systems Foundation2024Sáez de Ocáriz Borde, HArroyo, ÁMorales López, IPosner, IDong, XRecent research indicates that the performance of machine learning models can be improved by aligning the geometry of the latent space with the underlying data structure. Rather than relying solely on Euclidean space, researchers have proposed using hyperbolic and spherical spaces with constant curvature, or combinations thereof, to better model the latent space and enhance model performance. However, little attention has been given to the problem of automatically identifying the optimal latent geometry for the downstream task. We mathematically define this novel formulation and coin it as neural latent geometry search (NLGS). More specifically, we introduce an initial attempt to search for a latent geometry composed of a product of constant curvature model spaces with a small number of query evaluations, under some simplifying assumptions. To accomplish this, we propose a novel notion of distance between candidate latent geometries based on the Gromov-Hausdorff distance from metric geometry. In order to compute the Gromov-Hausdorff distance, we introduce a mapping function that enables the comparison of different manifolds by embedding them in a common high-dimensional ambient space. We then design a graph search space based on the notion of smoothness between latent geometries and employ the calculated distances as an additional inductive bias. Finally, we use Bayesian optimization to search for the optimal latent geometry in a query-efficient manner. This is a general method which can be applied to search for the optimal latent geometry for a variety of models and downstream tasks. We perform experiments on synthetic and real-world datasets to identify the optimal latent geometry for multiple machine learning problems. |
| spellingShingle | Sáez de Ocáriz Borde, H Arroyo, Á Morales López, I Posner, I Dong, X Neural latent geometry search: product manifold inference via Gromov-Hausdorff-informed Bayesian optimization |
| title | Neural latent geometry search: product manifold inference via Gromov-Hausdorff-informed Bayesian optimization
|
| title_full | Neural latent geometry search: product manifold inference via Gromov-Hausdorff-informed Bayesian optimization
|
| title_fullStr | Neural latent geometry search: product manifold inference via Gromov-Hausdorff-informed Bayesian optimization
|
| title_full_unstemmed | Neural latent geometry search: product manifold inference via Gromov-Hausdorff-informed Bayesian optimization
|
| title_short | Neural latent geometry search: product manifold inference via Gromov-Hausdorff-informed Bayesian optimization
|
| title_sort | neural latent geometry search product manifold inference via gromov hausdorff informed bayesian optimization |
| work_keys_str_mv | AT saezdeocarizbordeh neurallatentgeometrysearchproductmanifoldinferenceviagromovhausdorffinformedbayesianoptimization AT arroyoa neurallatentgeometrysearchproductmanifoldinferenceviagromovhausdorffinformedbayesianoptimization AT moraleslopezi neurallatentgeometrysearchproductmanifoldinferenceviagromovhausdorffinformedbayesianoptimization AT posneri neurallatentgeometrysearchproductmanifoldinferenceviagromovhausdorffinformedbayesianoptimization AT dongx neurallatentgeometrysearchproductmanifoldinferenceviagromovhausdorffinformedbayesianoptimization |