A Linearly Involved Generalized Moreau Enhancement of <i>ℓ</i><sub>2,1</sub>-Norm with Application to Weighted Group Sparse Classification
This paper proposes a new group-sparsity-inducing regularizer to approximate <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><msub><mo>ℓ</mo><mrow><mn>2</mn><mo>,</mo>&l...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2021-10-01
|
| Series: | Algorithms |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1999-4893/14/11/312 |
| _version_ | 1797511565977583616 |
|---|---|
| author | Yang Chen Masao Yamagishi Isao Yamada |
| author_facet | Yang Chen Masao Yamagishi Isao Yamada |
| author_sort | Yang Chen |
| collection | DOAJ |
| description | This paper proposes a new group-sparsity-inducing regularizer to approximate <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><msub><mo>ℓ</mo><mrow><mn>2</mn><mo>,</mo><mn>0</mn></mrow></msub></semantics></math></inline-formula> pseudo-norm. The regularizer is nonconvex, which can be seen as a linearly involved generalized Moreau enhancement of <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><msub><mo>ℓ</mo><mrow><mn>2</mn><mo>,</mo><mn>1</mn></mrow></msub></semantics></math></inline-formula>-norm. Moreover, the overall convexity of the corresponding group-sparsity-regularized least squares problem can be achieved. The model can handle general group configurations such as weighted group sparse problems, and can be solved through a proximal splitting algorithm. Among the applications, considering that the bias of convex regularizer may lead to incorrect classification results especially for unbalanced training sets, we apply the proposed model to the (weighted) group sparse classification problem. The proposed classifier can use the label, similarity and locality information of samples. It also suppresses the bias of convex regularizer-based classifiers. Experimental results demonstrate that the proposed classifier improves the performance of convex <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><msub><mo>ℓ</mo><mrow><mn>2</mn><mo>,</mo><mn>1</mn></mrow></msub></semantics></math></inline-formula> regularizer-based methods, especially when the training data set is unbalanced. This paper enhances the potential applicability and effectiveness of using nonconvex regularizers in the frame of convex optimization. |
| first_indexed | 2024-03-10T05:47:04Z |
| format | Article |
| id | doaj.art-64f9e46ca8f4408a91b5946d0094eaf3 |
| institution | Directory Open Access Journal |
| issn | 1999-4893 |
| language | English |
| last_indexed | 2024-03-10T05:47:04Z |
| publishDate | 2021-10-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Algorithms |
| spelling | doaj.art-64f9e46ca8f4408a91b5946d0094eaf32023-11-22T22:04:43ZengMDPI AGAlgorithms1999-48932021-10-01141131210.3390/a14110312A Linearly Involved Generalized Moreau Enhancement of <i>ℓ</i><sub>2,1</sub>-Norm with Application to Weighted Group Sparse ClassificationYang Chen0Masao Yamagishi1Isao Yamada2Department of Information and Communications Engineering, Tokyo Institute of Technology, 2-12-1 Okayama, Meguro-ku, Tokyo 152-8552, JapanDepartment of Information and Communications Engineering, Tokyo Institute of Technology, 2-12-1 Okayama, Meguro-ku, Tokyo 152-8552, JapanDepartment of Information and Communications Engineering, Tokyo Institute of Technology, 2-12-1 Okayama, Meguro-ku, Tokyo 152-8552, JapanThis paper proposes a new group-sparsity-inducing regularizer to approximate <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><msub><mo>ℓ</mo><mrow><mn>2</mn><mo>,</mo><mn>0</mn></mrow></msub></semantics></math></inline-formula> pseudo-norm. The regularizer is nonconvex, which can be seen as a linearly involved generalized Moreau enhancement of <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><msub><mo>ℓ</mo><mrow><mn>2</mn><mo>,</mo><mn>1</mn></mrow></msub></semantics></math></inline-formula>-norm. Moreover, the overall convexity of the corresponding group-sparsity-regularized least squares problem can be achieved. The model can handle general group configurations such as weighted group sparse problems, and can be solved through a proximal splitting algorithm. Among the applications, considering that the bias of convex regularizer may lead to incorrect classification results especially for unbalanced training sets, we apply the proposed model to the (weighted) group sparse classification problem. The proposed classifier can use the label, similarity and locality information of samples. It also suppresses the bias of convex regularizer-based classifiers. Experimental results demonstrate that the proposed classifier improves the performance of convex <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><msub><mo>ℓ</mo><mrow><mn>2</mn><mo>,</mo><mn>1</mn></mrow></msub></semantics></math></inline-formula> regularizer-based methods, especially when the training data set is unbalanced. This paper enhances the potential applicability and effectiveness of using nonconvex regularizers in the frame of convex optimization.https://www.mdpi.com/1999-4893/14/11/312convex optimizationproximal splitting algorithmgeneralized Moreau enhancementgroup sparsityweighted <i>ℓ</i><sub>2,1</sub>-normsparse representation-based classification |
| spellingShingle | Yang Chen Masao Yamagishi Isao Yamada A Linearly Involved Generalized Moreau Enhancement of <i>ℓ</i><sub>2,1</sub>-Norm with Application to Weighted Group Sparse Classification Algorithms convex optimization proximal splitting algorithm generalized Moreau enhancement group sparsity weighted <i>ℓ</i><sub>2,1</sub>-norm sparse representation-based classification |
| title | A Linearly Involved Generalized Moreau Enhancement of <i>ℓ</i><sub>2,1</sub>-Norm with Application to Weighted Group Sparse Classification |
| title_full | A Linearly Involved Generalized Moreau Enhancement of <i>ℓ</i><sub>2,1</sub>-Norm with Application to Weighted Group Sparse Classification |
| title_fullStr | A Linearly Involved Generalized Moreau Enhancement of <i>ℓ</i><sub>2,1</sub>-Norm with Application to Weighted Group Sparse Classification |
| title_full_unstemmed | A Linearly Involved Generalized Moreau Enhancement of <i>ℓ</i><sub>2,1</sub>-Norm with Application to Weighted Group Sparse Classification |
| title_short | A Linearly Involved Generalized Moreau Enhancement of <i>ℓ</i><sub>2,1</sub>-Norm with Application to Weighted Group Sparse Classification |
| title_sort | linearly involved generalized moreau enhancement of i l i sub 2 1 sub norm with application to weighted group sparse classification |
| topic | convex optimization proximal splitting algorithm generalized Moreau enhancement group sparsity weighted <i>ℓ</i><sub>2,1</sub>-norm sparse representation-based classification |
| url | https://www.mdpi.com/1999-4893/14/11/312 |
| work_keys_str_mv | AT yangchen alinearlyinvolvedgeneralizedmoreauenhancementofilisub21subnormwithapplicationtoweightedgroupsparseclassification AT masaoyamagishi alinearlyinvolvedgeneralizedmoreauenhancementofilisub21subnormwithapplicationtoweightedgroupsparseclassification AT isaoyamada alinearlyinvolvedgeneralizedmoreauenhancementofilisub21subnormwithapplicationtoweightedgroupsparseclassification AT yangchen linearlyinvolvedgeneralizedmoreauenhancementofilisub21subnormwithapplicationtoweightedgroupsparseclassification AT masaoyamagishi linearlyinvolvedgeneralizedmoreauenhancementofilisub21subnormwithapplicationtoweightedgroupsparseclassification AT isaoyamada linearlyinvolvedgeneralizedmoreauenhancementofilisub21subnormwithapplicationtoweightedgroupsparseclassification |