A novel quasi-Newton method for composite convex minimization
A fast parallelable Jacobi iteration type optimization method for non-smooth convex composite optimization is presented. Traditional gradient-based techniques cannot solve the problem. Smooth approximate functions are attempted to be used as a replacement of those non-smooth terms without compromisi...
| Main Authors: | , , |
|---|---|
| Other Authors: | |
| Format: | Journal Article |
| Language: | English |
| Published: |
2022
|
| Subjects: | |
| Online Access: | https://hdl.handle.net/10356/161093 |
| _version_ | 1811682518854795264 |
|---|---|
| author | Chai, Woon Huei Ho, Shen-Shyang Quek, Hiok Chai |
| author2 | School of Computer Science and Engineering |
| author_facet | School of Computer Science and Engineering Chai, Woon Huei Ho, Shen-Shyang Quek, Hiok Chai |
| author_sort | Chai, Woon Huei |
| collection | NTU |
| description | A fast parallelable Jacobi iteration type optimization method for non-smooth convex composite optimization is presented. Traditional gradient-based techniques cannot solve the problem. Smooth approximate functions are attempted to be used as a replacement of those non-smooth terms without compromising the accuracy. Recently, proximal mapping concept has been introduced into this field. Techniques which utilize proximal average based proximal gradient have been used to solve the problem. The state-of-art methods only utilize first-order information of the smooth approximate function. We integrate both first and second-order techniques to use both first and second-order information to boost the convergence speed. A convergence rate with a lower bound of O([Formula presented]) is achieved by the proposed method and a super-linear convergence is enjoyed when there is proper second-order information. In experiments, the proposed method converges significantly better than the state of art methods which enjoy O([Formula presented]) convergence. |
| first_indexed | 2024-10-01T03:58:07Z |
| format | Journal Article |
| id | ntu-10356/161093 |
| institution | Nanyang Technological University |
| language | English |
| last_indexed | 2024-10-01T03:58:07Z |
| publishDate | 2022 |
| record_format | dspace |
| spelling | ntu-10356/1610932022-08-15T07:04:55Z A novel quasi-Newton method for composite convex minimization Chai, Woon Huei Ho, Shen-Shyang Quek, Hiok Chai School of Computer Science and Engineering Interdisciplinary Graduate School (IGS) Energy Research Institute @ NTU (ERI@N) Rolls-Royce@NTU Corporate Lab Engineering::Computer science and engineering Proximal Mapping Quasi-Newton A fast parallelable Jacobi iteration type optimization method for non-smooth convex composite optimization is presented. Traditional gradient-based techniques cannot solve the problem. Smooth approximate functions are attempted to be used as a replacement of those non-smooth terms without compromising the accuracy. Recently, proximal mapping concept has been introduced into this field. Techniques which utilize proximal average based proximal gradient have been used to solve the problem. The state-of-art methods only utilize first-order information of the smooth approximate function. We integrate both first and second-order techniques to use both first and second-order information to boost the convergence speed. A convergence rate with a lower bound of O([Formula presented]) is achieved by the proposed method and a super-linear convergence is enjoyed when there is proper second-order information. In experiments, the proposed method converges significantly better than the state of art methods which enjoy O([Formula presented]) convergence. National Research Foundation (NRF) This work was conducted within the Rolls-Royce@NTU Corporate Lab with support from the National Research Foundation (NRF) Singapore un-der the Corp Lab@University Scheme and Energy Research Institute@NTU under Interdisciplinary Graduate School in Nanyang Technological University. 2022-08-15T07:04:55Z 2022-08-15T07:04:55Z 2022 Journal Article Chai, W. H., Ho, S. & Quek, H. C. (2022). A novel quasi-Newton method for composite convex minimization. Pattern Recognition, 122, 108281-. https://dx.doi.org/10.1016/j.patcog.2021.108281 0031-3203 https://hdl.handle.net/10356/161093 10.1016/j.patcog.2021.108281 2-s2.0-85120963556 122 108281 en Pattern Recognition © 2021 Elsevier Ltd. All rights reserved. |
| spellingShingle | Engineering::Computer science and engineering Proximal Mapping Quasi-Newton Chai, Woon Huei Ho, Shen-Shyang Quek, Hiok Chai A novel quasi-Newton method for composite convex minimization |
| title | A novel quasi-Newton method for composite convex minimization |
| title_full | A novel quasi-Newton method for composite convex minimization |
| title_fullStr | A novel quasi-Newton method for composite convex minimization |
| title_full_unstemmed | A novel quasi-Newton method for composite convex minimization |
| title_short | A novel quasi-Newton method for composite convex minimization |
| title_sort | novel quasi newton method for composite convex minimization |
| topic | Engineering::Computer science and engineering Proximal Mapping Quasi-Newton |
| url | https://hdl.handle.net/10356/161093 |
| work_keys_str_mv | AT chaiwoonhuei anovelquasinewtonmethodforcompositeconvexminimization AT hoshenshyang anovelquasinewtonmethodforcompositeconvexminimization AT quekhiokchai anovelquasinewtonmethodforcompositeconvexminimization AT chaiwoonhuei novelquasinewtonmethodforcompositeconvexminimization AT hoshenshyang novelquasinewtonmethodforcompositeconvexminimization AT quekhiokchai novelquasinewtonmethodforcompositeconvexminimization |