Robust classification with feature selection using an application of the Douglas-Rachford splitting algorithm
This paper deals with supervised classification and feature selection with application in the context of high dimensional features. A classical approach leads to an optimization problem minimizing the within sum of squares in the clusters (I2 norm) with an I1 penalty in order to promote sparsity. It...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
EDP Sciences
2021-08-01
|
| Series: | ESAIM: Proceedings and Surveys |
| Online Access: | https://www.esaim-proc.org/articles/proc/pdf/2021/02/proc2107102.pdf |
| _version_ | 1797966688905330688 |
|---|---|
| author | Barlaud Michel Antonini Marc |
| author_facet | Barlaud Michel Antonini Marc |
| author_sort | Barlaud Michel |
| collection | DOAJ |
| description | This paper deals with supervised classification and feature selection with application in the context of high dimensional features. A classical approach leads to an optimization problem minimizing the within sum of squares in the clusters (I2 norm) with an I1 penalty in order to promote sparsity. It has been known for decades that I1 norm is more robust than I2 norm to outliers. In this paper, we deal with this issue using a new proximal splitting method for the minimization of a criterion using I2 norm both for the constraint and the loss function. Since the I1 criterion is only convex and not gradient Lipschitz, we advocate the use of a Douglas-Rachford minimization solution. We take advantage of the particular form of the cost and, using a change of variable, we provide a new efficient tailored primal Douglas-Rachford splitting algorithm which is very effective on high dimensional dataset. We also provide an efficient classifier in the projected space based on medoid modeling. Experiments on two biological datasets and a computer vision dataset show that our method significantly improves the results compared to those obtained using a quadratic loss function. |
| first_indexed | 2024-04-11T02:19:17Z |
| format | Article |
| id | doaj.art-557c669acd784cc9905875777bb637ce |
| institution | Directory Open Access Journal |
| issn | 2267-3059 |
| language | English |
| last_indexed | 2024-04-11T02:19:17Z |
| publishDate | 2021-08-01 |
| publisher | EDP Sciences |
| record_format | Article |
| series | ESAIM: Proceedings and Surveys |
| spelling | doaj.art-557c669acd784cc9905875777bb637ce2023-01-03T00:20:33ZengEDP SciencesESAIM: Proceedings and Surveys2267-30592021-08-0171112010.1051/proc/202171102proc2107102Robust classification with feature selection using an application of the Douglas-Rachford splitting algorithmBarlaud Michel0Antonini Marc1I3S, Univ. Côte d’Azur & CNRSI3S, Univ. Côte d’Azur & CNRSThis paper deals with supervised classification and feature selection with application in the context of high dimensional features. A classical approach leads to an optimization problem minimizing the within sum of squares in the clusters (I2 norm) with an I1 penalty in order to promote sparsity. It has been known for decades that I1 norm is more robust than I2 norm to outliers. In this paper, we deal with this issue using a new proximal splitting method for the minimization of a criterion using I2 norm both for the constraint and the loss function. Since the I1 criterion is only convex and not gradient Lipschitz, we advocate the use of a Douglas-Rachford minimization solution. We take advantage of the particular form of the cost and, using a change of variable, we provide a new efficient tailored primal Douglas-Rachford splitting algorithm which is very effective on high dimensional dataset. We also provide an efficient classifier in the projected space based on medoid modeling. Experiments on two biological datasets and a computer vision dataset show that our method significantly improves the results compared to those obtained using a quadratic loss function.https://www.esaim-proc.org/articles/proc/pdf/2021/02/proc2107102.pdf |
| spellingShingle | Barlaud Michel Antonini Marc Robust classification with feature selection using an application of the Douglas-Rachford splitting algorithm ESAIM: Proceedings and Surveys |
| title | Robust classification with feature selection using an application of the Douglas-Rachford splitting algorithm |
| title_full | Robust classification with feature selection using an application of the Douglas-Rachford splitting algorithm |
| title_fullStr | Robust classification with feature selection using an application of the Douglas-Rachford splitting algorithm |
| title_full_unstemmed | Robust classification with feature selection using an application of the Douglas-Rachford splitting algorithm |
| title_short | Robust classification with feature selection using an application of the Douglas-Rachford splitting algorithm |
| title_sort | robust classification with feature selection using an application of the douglas rachford splitting algorithm |
| url | https://www.esaim-proc.org/articles/proc/pdf/2021/02/proc2107102.pdf |
| work_keys_str_mv | AT barlaudmichel robustclassificationwithfeatureselectionusinganapplicationofthedouglasrachfordsplittingalgorithm AT antoninimarc robustclassificationwithfeatureselectionusinganapplicationofthedouglasrachfordsplittingalgorithm |