Proximal variable metric method with spectral diagonal update for large scale sparse optimization
We will tackle the l0-norm sparse optimization problem using an underdetermined system as a constraint in this research. This problem is turned into an unconstrained optimization problem using the Lagrangian method and solved using the proximal variable metric method. This approach combines the p...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Published: |
Elsevier
2023
|
| _version_ | 1825939695143485440 |
|---|---|
| author | Woo, Gillian Yi Han Sim, Hong Seng Goh, Yong Kheng Leong, Wah June |
| author_facet | Woo, Gillian Yi Han Sim, Hong Seng Goh, Yong Kheng Leong, Wah June |
| author_sort | Woo, Gillian Yi Han |
| collection | UPM |
| description | We will tackle the l0-norm sparse optimization problem using an underdetermined system as a
constraint in this research. This problem is turned into an unconstrained optimization problem using
the Lagrangian method and solved using the proximal variable metric method. This approach combines
the proximal and variable metric methods by substituting a diagonal matrix for the approximation of
the full rank Hessian matrix. Hence, the memory requirement is reduced to O(n) storage instead of
O(n2 ) storage. The diagonal updating matrix is derived from the same variational technique used in
the derivation of variable metric or quasi-Newton updates but incorporated with some weaker form
of quasi-Newton relation. Convergence analysis of this method is established. The efficiency of the
proposed method is compared against existing versions of proximal gradient methods on simulated
datasets and large real-world MNIST datasets using Python software. Numerical results show that our
proposed method is more robust and stable for finding sparse solutions to the linear system. |
| first_indexed | 2024-12-09T02:21:07Z |
| format | Article |
| id | upm.eprints-109005 |
| institution | Universiti Putra Malaysia |
| last_indexed | 2024-12-09T02:21:07Z |
| publishDate | 2023 |
| publisher | Elsevier |
| record_format | dspace |
| spelling | upm.eprints-1090052024-10-14T06:59:42Z http://psasir.upm.edu.my/id/eprint/109005/ Proximal variable metric method with spectral diagonal update for large scale sparse optimization Woo, Gillian Yi Han Sim, Hong Seng Goh, Yong Kheng Leong, Wah June We will tackle the l0-norm sparse optimization problem using an underdetermined system as a constraint in this research. This problem is turned into an unconstrained optimization problem using the Lagrangian method and solved using the proximal variable metric method. This approach combines the proximal and variable metric methods by substituting a diagonal matrix for the approximation of the full rank Hessian matrix. Hence, the memory requirement is reduced to O(n) storage instead of O(n2 ) storage. The diagonal updating matrix is derived from the same variational technique used in the derivation of variable metric or quasi-Newton updates but incorporated with some weaker form of quasi-Newton relation. Convergence analysis of this method is established. The efficiency of the proposed method is compared against existing versions of proximal gradient methods on simulated datasets and large real-world MNIST datasets using Python software. Numerical results show that our proposed method is more robust and stable for finding sparse solutions to the linear system. Elsevier 2023-05 Article PeerReviewed Woo, Gillian Yi Han and Sim, Hong Seng and Goh, Yong Kheng and Leong, Wah June (2023) Proximal variable metric method with spectral diagonal update for large scale sparse optimization. Journal of the Franklin Institute, 360 (7). pp. 4640-4660. ISSN 0016-0032; ESSN: 1879-2693 https://linkinghub.elsevier.com/retrieve/pii/S001600322300145X 10.1016/j.jfranklin.2023.02.035 |
| spellingShingle | Woo, Gillian Yi Han Sim, Hong Seng Goh, Yong Kheng Leong, Wah June Proximal variable metric method with spectral diagonal update for large scale sparse optimization |
| title | Proximal variable metric method with spectral diagonal update for large scale sparse optimization |
| title_full | Proximal variable metric method with spectral diagonal update for large scale sparse optimization |
| title_fullStr | Proximal variable metric method with spectral diagonal update for large scale sparse optimization |
| title_full_unstemmed | Proximal variable metric method with spectral diagonal update for large scale sparse optimization |
| title_short | Proximal variable metric method with spectral diagonal update for large scale sparse optimization |
| title_sort | proximal variable metric method with spectral diagonal update for large scale sparse optimization |
| work_keys_str_mv | AT woogillianyihan proximalvariablemetricmethodwithspectraldiagonalupdateforlargescalesparseoptimization AT simhongseng proximalvariablemetricmethodwithspectraldiagonalupdateforlargescalesparseoptimization AT gohyongkheng proximalvariablemetricmethodwithspectraldiagonalupdateforlargescalesparseoptimization AT leongwahjune proximalvariablemetricmethodwithspectraldiagonalupdateforlargescalesparseoptimization |