Quasi-Newton methods based on ordinary differential equation approach for unconstrained nonlinear optimization
In this paper, we propose a hybrid ODE-based quasi-Newton (QN) method for unconstrained optimization problems, which combines the idea of low-order implicit Runge–Kutta (RK) techniques for gradient systems with the QN type updates of the Jacobian matrix such as the symmetric rank-one (SR1) update. T...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Published: |
Elsevier Inc.
2014
|
| _version_ | 1825948587770511360 |
|---|---|
| author | Khiyabani, Farzin Modarres Leong, Wah June |
| author_facet | Khiyabani, Farzin Modarres Leong, Wah June |
| author_sort | Khiyabani, Farzin Modarres |
| collection | UPM |
| description | In this paper, we propose a hybrid ODE-based quasi-Newton (QN) method for unconstrained optimization problems, which combines the idea of low-order implicit Runge–Kutta (RK) techniques for gradient systems with the QN type updates of the Jacobian matrix such as the symmetric rank-one (SR1) update. The main idea of this approach is to associate a QN matrix to approximate numerically the Jacobian matrix in the gradient system. Fundamentally this is a gradient system based on the first order optimality conditions of the optimization problem. To further extend the methods for solving large-scale problems, a feature incorporated to the proposed methods is that a limited memory setting for matrix–vector multiplications is used, thus avoiding the computational and storage issues when computing Jacobian information. Under suitable assumptions, global convergence of the proposed method is proved. Practical insights on the effectiveness of these approaches on a set of test functions are given by a numerical comparison with that of the limited memory BFGS algorithm (L-BFGS) and conjugate gradient algorithm (CG). |
| first_indexed | 2024-03-06T08:31:24Z |
| format | Article |
| id | upm.eprints-35140 |
| institution | Universiti Putra Malaysia |
| last_indexed | 2024-03-06T08:31:24Z |
| publishDate | 2014 |
| publisher | Elsevier Inc. |
| record_format | dspace |
| spelling | upm.eprints-351402016-01-14T03:41:57Z http://psasir.upm.edu.my/id/eprint/35140/ Quasi-Newton methods based on ordinary differential equation approach for unconstrained nonlinear optimization Khiyabani, Farzin Modarres Leong, Wah June In this paper, we propose a hybrid ODE-based quasi-Newton (QN) method for unconstrained optimization problems, which combines the idea of low-order implicit Runge–Kutta (RK) techniques for gradient systems with the QN type updates of the Jacobian matrix such as the symmetric rank-one (SR1) update. The main idea of this approach is to associate a QN matrix to approximate numerically the Jacobian matrix in the gradient system. Fundamentally this is a gradient system based on the first order optimality conditions of the optimization problem. To further extend the methods for solving large-scale problems, a feature incorporated to the proposed methods is that a limited memory setting for matrix–vector multiplications is used, thus avoiding the computational and storage issues when computing Jacobian information. Under suitable assumptions, global convergence of the proposed method is proved. Practical insights on the effectiveness of these approaches on a set of test functions are given by a numerical comparison with that of the limited memory BFGS algorithm (L-BFGS) and conjugate gradient algorithm (CG). Elsevier Inc. 2014-05-01 Article PeerReviewed Khiyabani, Farzin Modarres and Leong, Wah June (2014) Quasi-Newton methods based on ordinary differential equation approach for unconstrained nonlinear optimization. Applied Mathematics and Computation, 233. pp. 272-291. ISSN 0096-3003; ESSN: 1873-5649 http://www.sciencedirect.com/science/journal/00963003/233/supp/C 10.1016/j.amc.2014.01.171 |
| spellingShingle | Khiyabani, Farzin Modarres Leong, Wah June Quasi-Newton methods based on ordinary differential equation approach for unconstrained nonlinear optimization |
| title | Quasi-Newton methods based on ordinary differential equation approach for unconstrained nonlinear optimization |
| title_full | Quasi-Newton methods based on ordinary differential equation approach for unconstrained nonlinear optimization |
| title_fullStr | Quasi-Newton methods based on ordinary differential equation approach for unconstrained nonlinear optimization |
| title_full_unstemmed | Quasi-Newton methods based on ordinary differential equation approach for unconstrained nonlinear optimization |
| title_short | Quasi-Newton methods based on ordinary differential equation approach for unconstrained nonlinear optimization |
| title_sort | quasi newton methods based on ordinary differential equation approach for unconstrained nonlinear optimization |
| work_keys_str_mv | AT khiyabanifarzinmodarres quasinewtonmethodsbasedonordinarydifferentialequationapproachforunconstrainednonlinearoptimization AT leongwahjune quasinewtonmethodsbasedonordinarydifferentialequationapproachforunconstrainednonlinearoptimization |