A Novel Robust Metric Distance Optimization-Driven Manifold Learning Framework for Semi-Supervised Pattern Classification
In this work, we address the problem of improving the classification performance of machine learning models, especially in the presence of noisy and outlier data. To this end, we first innovatively design a generalized adaptive robust loss function called <inline-formula><math xmlns="h...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-07-01
|
| Series: | Axioms |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2075-1680/12/8/737 |
| _version_ | 1797585521781768192 |
|---|---|
| author | Bao Ma Jun Ma Guolin Yu |
| author_facet | Bao Ma Jun Ma Guolin Yu |
| author_sort | Bao Ma |
| collection | DOAJ |
| description | In this work, we address the problem of improving the classification performance of machine learning models, especially in the presence of noisy and outlier data. To this end, we first innovatively design a generalized adaptive robust loss function called <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mrow><msub><mi>V</mi><mi>θ</mi></msub><mrow><mo stretchy="false">(</mo><mi>x</mi><mo stretchy="false">)</mo></mrow></mrow></semantics></math></inline-formula>. Intuitively, <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mrow><msub><mi>V</mi><mi>θ</mi></msub><mrow><mo stretchy="false">(</mo><mi>x</mi><mo stretchy="false">)</mo></mrow></mrow></semantics></math></inline-formula> can improve the robustness of the model by selecting different robust loss functions for different learning tasks during the learning process via the adaptive parameter <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mi>θ</mi></semantics></math></inline-formula>. Compared with other robust loss functions, <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mrow><msub><mi>V</mi><mi>θ</mi></msub><mrow><mo stretchy="false">(</mo><mi>x</mi><mo stretchy="false">)</mo></mrow></mrow></semantics></math></inline-formula> has some desirable salient properties, such as symmetry, boundedness, robustness, nonconvexity, and adaptivity, making it suitable for a wide range of machine learning applications. Secondly, a new robust semi-supervised learning framework for pattern classification is proposed. In this learning framework, the proposed robust loss function <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mrow><msub><mi>V</mi><mi>θ</mi></msub><mrow><mo stretchy="false">(</mo><mi>x</mi><mo stretchy="false">)</mo></mrow></mrow></semantics></math></inline-formula> and capped <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><msub><mi>L</mi><mrow><mn>2</mn><mo>,</mo><mi>p</mi></mrow></msub></semantics></math></inline-formula>-norm robust distance metric are introduced to improve the robustness and generalization performance of the model, especially when the outliers are far from the normal data distributions. Simultaneously, based on this learning framework, the Welsch manifold robust twin bounded support vector machine (WMRTBSVM) and its least-squares version are developed. Finally, two effective iterative optimization algorithms are designed, their convergence is proved, and their complexity is calculated. Experimental results on several datasets with different noise settings and different evaluation criteria show that our methods have better classification performance and robustness. With the Cancer dataset, when there is no noise, the classification accuracy of our proposed methods is <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mrow><mn>94.17</mn><mo>%</mo></mrow></semantics></math></inline-formula> and <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mrow><mn>95.62</mn><mo>%</mo></mrow></semantics></math></inline-formula>, respectively. When the Gaussian noise is <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mrow><mn>50</mn><mo>%</mo></mrow></semantics></math></inline-formula>, the classification accuracy of our proposed methods is <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mrow><mn>91.76</mn><mo>%</mo></mrow></semantics></math></inline-formula> and <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mrow><mn>90.59</mn><mo>%</mo></mrow></semantics></math></inline-formula>, respectively, demonstrating that our method has satisfactory classification performance and robustness. |
| first_indexed | 2024-03-11T00:07:19Z |
| format | Article |
| id | doaj.art-468e55dc5c9c494a8ced975914158f1a |
| institution | Directory Open Access Journal |
| issn | 2075-1680 |
| language | English |
| last_indexed | 2024-03-11T00:07:19Z |
| publishDate | 2023-07-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Axioms |
| spelling | doaj.art-468e55dc5c9c494a8ced975914158f1a2023-11-19T00:14:27ZengMDPI AGAxioms2075-16802023-07-0112873710.3390/axioms12080737A Novel Robust Metric Distance Optimization-Driven Manifold Learning Framework for Semi-Supervised Pattern ClassificationBao Ma0Jun Ma1Guolin Yu2School of Mathematics and Information Sciences, North Minzu University, Yinchuan 750021, ChinaSchool of Mathematics and Information Sciences, North Minzu University, Yinchuan 750021, ChinaSchool of Mathematics and Information Sciences, North Minzu University, Yinchuan 750021, ChinaIn this work, we address the problem of improving the classification performance of machine learning models, especially in the presence of noisy and outlier data. To this end, we first innovatively design a generalized adaptive robust loss function called <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mrow><msub><mi>V</mi><mi>θ</mi></msub><mrow><mo stretchy="false">(</mo><mi>x</mi><mo stretchy="false">)</mo></mrow></mrow></semantics></math></inline-formula>. Intuitively, <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mrow><msub><mi>V</mi><mi>θ</mi></msub><mrow><mo stretchy="false">(</mo><mi>x</mi><mo stretchy="false">)</mo></mrow></mrow></semantics></math></inline-formula> can improve the robustness of the model by selecting different robust loss functions for different learning tasks during the learning process via the adaptive parameter <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mi>θ</mi></semantics></math></inline-formula>. Compared with other robust loss functions, <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mrow><msub><mi>V</mi><mi>θ</mi></msub><mrow><mo stretchy="false">(</mo><mi>x</mi><mo stretchy="false">)</mo></mrow></mrow></semantics></math></inline-formula> has some desirable salient properties, such as symmetry, boundedness, robustness, nonconvexity, and adaptivity, making it suitable for a wide range of machine learning applications. Secondly, a new robust semi-supervised learning framework for pattern classification is proposed. In this learning framework, the proposed robust loss function <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mrow><msub><mi>V</mi><mi>θ</mi></msub><mrow><mo stretchy="false">(</mo><mi>x</mi><mo stretchy="false">)</mo></mrow></mrow></semantics></math></inline-formula> and capped <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><msub><mi>L</mi><mrow><mn>2</mn><mo>,</mo><mi>p</mi></mrow></msub></semantics></math></inline-formula>-norm robust distance metric are introduced to improve the robustness and generalization performance of the model, especially when the outliers are far from the normal data distributions. Simultaneously, based on this learning framework, the Welsch manifold robust twin bounded support vector machine (WMRTBSVM) and its least-squares version are developed. Finally, two effective iterative optimization algorithms are designed, their convergence is proved, and their complexity is calculated. Experimental results on several datasets with different noise settings and different evaluation criteria show that our methods have better classification performance and robustness. With the Cancer dataset, when there is no noise, the classification accuracy of our proposed methods is <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mrow><mn>94.17</mn><mo>%</mo></mrow></semantics></math></inline-formula> and <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mrow><mn>95.62</mn><mo>%</mo></mrow></semantics></math></inline-formula>, respectively. When the Gaussian noise is <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mrow><mn>50</mn><mo>%</mo></mrow></semantics></math></inline-formula>, the classification accuracy of our proposed methods is <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mrow><mn>91.76</mn><mo>%</mo></mrow></semantics></math></inline-formula> and <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mrow><mn>90.59</mn><mo>%</mo></mrow></semantics></math></inline-formula>, respectively, demonstrating that our method has satisfactory classification performance and robustness.https://www.mdpi.com/2075-1680/12/8/737robust distance metricloss functionmanifold regularizationsemi-supervised learningpattern classification |
| spellingShingle | Bao Ma Jun Ma Guolin Yu A Novel Robust Metric Distance Optimization-Driven Manifold Learning Framework for Semi-Supervised Pattern Classification Axioms robust distance metric loss function manifold regularization semi-supervised learning pattern classification |
| title | A Novel Robust Metric Distance Optimization-Driven Manifold Learning Framework for Semi-Supervised Pattern Classification |
| title_full | A Novel Robust Metric Distance Optimization-Driven Manifold Learning Framework for Semi-Supervised Pattern Classification |
| title_fullStr | A Novel Robust Metric Distance Optimization-Driven Manifold Learning Framework for Semi-Supervised Pattern Classification |
| title_full_unstemmed | A Novel Robust Metric Distance Optimization-Driven Manifold Learning Framework for Semi-Supervised Pattern Classification |
| title_short | A Novel Robust Metric Distance Optimization-Driven Manifold Learning Framework for Semi-Supervised Pattern Classification |
| title_sort | novel robust metric distance optimization driven manifold learning framework for semi supervised pattern classification |
| topic | robust distance metric loss function manifold regularization semi-supervised learning pattern classification |
| url | https://www.mdpi.com/2075-1680/12/8/737 |
| work_keys_str_mv | AT baoma anovelrobustmetricdistanceoptimizationdrivenmanifoldlearningframeworkforsemisupervisedpatternclassification AT junma anovelrobustmetricdistanceoptimizationdrivenmanifoldlearningframeworkforsemisupervisedpatternclassification AT guolinyu anovelrobustmetricdistanceoptimizationdrivenmanifoldlearningframeworkforsemisupervisedpatternclassification AT baoma novelrobustmetricdistanceoptimizationdrivenmanifoldlearningframeworkforsemisupervisedpatternclassification AT junma novelrobustmetricdistanceoptimizationdrivenmanifoldlearningframeworkforsemisupervisedpatternclassification AT guolinyu novelrobustmetricdistanceoptimizationdrivenmanifoldlearningframeworkforsemisupervisedpatternclassification |