Objective acceleration for unconstrained optimization
Acceleration schemes can dramatically improve existing optimization procedures. In most of the work on these schemes, such as nonlinear Generalized Minimal Residual (N-GMRES), acceleration is based on minimizing the $\ell_2$ norm of some target on subspaces of $\mathbb{R}^n$. There are many numerica...
| Egile nagusia: | |
|---|---|
| Formatua: | Journal article |
| Argitaratua: |
Wiley
2018
|
| _version_ | 1826291605304246272 |
|---|---|
| author | Riseth, A |
| author_facet | Riseth, A |
| author_sort | Riseth, A |
| collection | OXFORD |
| description | Acceleration schemes can dramatically improve existing optimization procedures. In most of the work on these schemes, such as nonlinear Generalized Minimal Residual (N-GMRES), acceleration is based on minimizing the $\ell_2$ norm of some target on subspaces of $\mathbb{R}^n$. There are many numerical examples that show how accelerating general purpose and domain-specific optimizers with N-GMRES results in large improvements. We propose a natural modification to N-GMRES, which significantly improves the performance in a testing environment originally used to advocate N-GMRES. Our proposed approach, which we refer to as O-ACCEL (Objective Acceleration), is novel in that it minimizes an approximation to the \emph{objective function} on subspaces of $\mathbb{R}^n$. We prove that O-ACCEL reduces to the Full Orthogonalization Method for linear systems when the objective is quadratic, which differentiates our proposed approach from existing acceleration methods. Comparisons with L-BFGS and N-CG indicate the competitiveness of O-ACCEL. As it can be combined with domain-specific optimizers, it may also be beneficial in areas where L-BFGS or N-CG are not suitable. |
| first_indexed | 2024-03-07T03:01:54Z |
| format | Journal article |
| id | oxford-uuid:b13d69ca-8b30-4ca3-8ef0-40a865cdbcc4 |
| institution | University of Oxford |
| last_indexed | 2024-03-07T03:01:54Z |
| publishDate | 2018 |
| publisher | Wiley |
| record_format | dspace |
| spelling | oxford-uuid:b13d69ca-8b30-4ca3-8ef0-40a865cdbcc42022-03-27T04:02:32ZObjective acceleration for unconstrained optimizationJournal articlehttp://purl.org/coar/resource_type/c_dcae04bcuuid:b13d69ca-8b30-4ca3-8ef0-40a865cdbcc4Symplectic Elements at OxfordWiley2018Riseth, AAcceleration schemes can dramatically improve existing optimization procedures. In most of the work on these schemes, such as nonlinear Generalized Minimal Residual (N-GMRES), acceleration is based on minimizing the $\ell_2$ norm of some target on subspaces of $\mathbb{R}^n$. There are many numerical examples that show how accelerating general purpose and domain-specific optimizers with N-GMRES results in large improvements. We propose a natural modification to N-GMRES, which significantly improves the performance in a testing environment originally used to advocate N-GMRES. Our proposed approach, which we refer to as O-ACCEL (Objective Acceleration), is novel in that it minimizes an approximation to the \emph{objective function} on subspaces of $\mathbb{R}^n$. We prove that O-ACCEL reduces to the Full Orthogonalization Method for linear systems when the objective is quadratic, which differentiates our proposed approach from existing acceleration methods. Comparisons with L-BFGS and N-CG indicate the competitiveness of O-ACCEL. As it can be combined with domain-specific optimizers, it may also be beneficial in areas where L-BFGS or N-CG are not suitable. |
| spellingShingle | Riseth, A Objective acceleration for unconstrained optimization |
| title | Objective acceleration for unconstrained optimization |
| title_full | Objective acceleration for unconstrained optimization |
| title_fullStr | Objective acceleration for unconstrained optimization |
| title_full_unstemmed | Objective acceleration for unconstrained optimization |
| title_short | Objective acceleration for unconstrained optimization |
| title_sort | objective acceleration for unconstrained optimization |
| work_keys_str_mv | AT risetha objectiveaccelerationforunconstrainedoptimization |