A comparison of mixed-variables Bayesian optimization approaches
Abstract Most real optimization problems are defined over a mixed search space where the variables are both discrete and continuous. In engineering applications, the objective function is typically calculated with a numerically costly black-box simulation. General mixed and costly optimization probl...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
SpringerOpen
2022-06-01
|
| Series: | Advanced Modeling and Simulation in Engineering Sciences |
| Online Access: | https://doi.org/10.1186/s40323-022-00218-8 |
| _version_ | 1811258130718261248 |
|---|---|
| author | Jhouben Cuesta Ramirez Rodolphe Le Riche Olivier Roustant Guillaume Perrin Cédric Durantin Alain Glière |
| author_facet | Jhouben Cuesta Ramirez Rodolphe Le Riche Olivier Roustant Guillaume Perrin Cédric Durantin Alain Glière |
| author_sort | Jhouben Cuesta Ramirez |
| collection | DOAJ |
| description | Abstract Most real optimization problems are defined over a mixed search space where the variables are both discrete and continuous. In engineering applications, the objective function is typically calculated with a numerically costly black-box simulation. General mixed and costly optimization problems are therefore of a great practical interest, yet their resolution remains in a large part an open scientific question. In this article, costly mixed problems are approached through Gaussian processes where the discrete variables are relaxed into continuous latent variables. The continuous space is more easily harvested by classical Bayesian optimization techniques than a mixed space would. Discrete variables are recovered either subsequently to the continuous optimization, or simultaneously with an additional continuous-discrete compatibility constraint that is handled with augmented Lagrangians. Several possible implementations of such Bayesian mixed optimizers are compared. In particular, the reformulation of the problem with continuous latent variables is put in competition with searches working directly in the mixed space. Among the algorithms involving latent variables and an augmented Lagrangian, a particular attention is devoted to the Lagrange multipliers for which a local and a global estimation techniques are studied. The comparisons are based on the repeated optimization of three analytical functions and a beam design problem. |
| first_indexed | 2024-04-12T18:08:26Z |
| format | Article |
| id | doaj.art-b30aba4e600049089d0c50cb307b110d |
| institution | Directory Open Access Journal |
| issn | 2213-7467 |
| language | English |
| last_indexed | 2024-04-12T18:08:26Z |
| publishDate | 2022-06-01 |
| publisher | SpringerOpen |
| record_format | Article |
| series | Advanced Modeling and Simulation in Engineering Sciences |
| spelling | doaj.art-b30aba4e600049089d0c50cb307b110d2022-12-22T03:21:55ZengSpringerOpenAdvanced Modeling and Simulation in Engineering Sciences2213-74672022-06-019112910.1186/s40323-022-00218-8A comparison of mixed-variables Bayesian optimization approachesJhouben Cuesta Ramirez0Rodolphe Le Riche1Olivier Roustant2Guillaume Perrin3Cédric Durantin4Alain Glière5CEA, LETI, Univ. Grenoble AlpesLIMOS (CNRS, Mines Saint-Etienne, UCA)INSA ToulouseCOSYS, Université Gustave EiffelCEA-DAMCEA, LETI, Univ. Grenoble AlpesAbstract Most real optimization problems are defined over a mixed search space where the variables are both discrete and continuous. In engineering applications, the objective function is typically calculated with a numerically costly black-box simulation. General mixed and costly optimization problems are therefore of a great practical interest, yet their resolution remains in a large part an open scientific question. In this article, costly mixed problems are approached through Gaussian processes where the discrete variables are relaxed into continuous latent variables. The continuous space is more easily harvested by classical Bayesian optimization techniques than a mixed space would. Discrete variables are recovered either subsequently to the continuous optimization, or simultaneously with an additional continuous-discrete compatibility constraint that is handled with augmented Lagrangians. Several possible implementations of such Bayesian mixed optimizers are compared. In particular, the reformulation of the problem with continuous latent variables is put in competition with searches working directly in the mixed space. Among the algorithms involving latent variables and an augmented Lagrangian, a particular attention is devoted to the Lagrange multipliers for which a local and a global estimation techniques are studied. The comparisons are based on the repeated optimization of three analytical functions and a beam design problem.https://doi.org/10.1186/s40323-022-00218-8 |
| spellingShingle | Jhouben Cuesta Ramirez Rodolphe Le Riche Olivier Roustant Guillaume Perrin Cédric Durantin Alain Glière A comparison of mixed-variables Bayesian optimization approaches Advanced Modeling and Simulation in Engineering Sciences |
| title | A comparison of mixed-variables Bayesian optimization approaches |
| title_full | A comparison of mixed-variables Bayesian optimization approaches |
| title_fullStr | A comparison of mixed-variables Bayesian optimization approaches |
| title_full_unstemmed | A comparison of mixed-variables Bayesian optimization approaches |
| title_short | A comparison of mixed-variables Bayesian optimization approaches |
| title_sort | comparison of mixed variables bayesian optimization approaches |
| url | https://doi.org/10.1186/s40323-022-00218-8 |
| work_keys_str_mv | AT jhoubencuestaramirez acomparisonofmixedvariablesbayesianoptimizationapproaches AT rodolpheleriche acomparisonofmixedvariablesbayesianoptimizationapproaches AT olivierroustant acomparisonofmixedvariablesbayesianoptimizationapproaches AT guillaumeperrin acomparisonofmixedvariablesbayesianoptimizationapproaches AT cedricdurantin acomparisonofmixedvariablesbayesianoptimizationapproaches AT alaingliere acomparisonofmixedvariablesbayesianoptimizationapproaches AT jhoubencuestaramirez comparisonofmixedvariablesbayesianoptimizationapproaches AT rodolpheleriche comparisonofmixedvariablesbayesianoptimizationapproaches AT olivierroustant comparisonofmixedvariablesbayesianoptimizationapproaches AT guillaumeperrin comparisonofmixedvariablesbayesianoptimizationapproaches AT cedricdurantin comparisonofmixedvariablesbayesianoptimizationapproaches AT alaingliere comparisonofmixedvariablesbayesianoptimizationapproaches |