OPT-RNN-DBSVM: OPTimal Recurrent Neural Network and Density-Based Support Vector Machine
When implementing SVMs, two major problems are encountered: (a) the number of local minima of dual-SVM increases exponentially with the number of samples and (b) the computer storage memory required for a regular quadratic programming solver increases exponentially as the problem size expands. The K...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-08-01
|
| Series: | Mathematics |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2227-7390/11/16/3555 |
| _version_ | 1797583958398992384 |
|---|---|
| author | Karim El Moutaouakil Abdellatif El Ouissari Adrian Olaru Vasile Palade Mihaela Ciorei |
| author_facet | Karim El Moutaouakil Abdellatif El Ouissari Adrian Olaru Vasile Palade Mihaela Ciorei |
| author_sort | Karim El Moutaouakil |
| collection | DOAJ |
| description | When implementing SVMs, two major problems are encountered: (a) the number of local minima of dual-SVM increases exponentially with the number of samples and (b) the computer storage memory required for a regular quadratic programming solver increases exponentially as the problem size expands. The Kernel-Adatron family of algorithms, gaining attention recently, has allowed us to handle very large classification and regression problems. However, these methods treat different types of samples (i.e., noise, border, and core) in the same manner, which makes these algorithms search in unpromising areas and increases the number of iterations as well. This paper introduces a hybrid method to overcome such shortcomings, called the Optimal Recurrent Neural Network and Density-Based Support Vector Machine (Opt-RNN-DBSVM). This method consists of four steps: (a) the characterization of different samples, (b) the elimination of samples with a low probability of being a support vector, (c) the construction of an appropriate recurrent neural network to solve the dual-DBSVM based on an original energy function, and (d) finding the solution to the system of differential equations that govern the dynamics of the RNN, using the Euler–Cauchy method involving an optimal time step. Density-based preprocessing reduces the number of local minima in the dual-SVM. The RNN’s recurring architecture avoids the need to explore recently visited areas. With the optimal time step, the search moves from the current vectors to the best neighboring support vectors. It is demonstrated that RNN-SVM converges to feasible support vectors and Opt-RNN-DBSVM has very low time complexity compared to the RNN-SVM with a constant time step and the Kernel-Adatron algorithm–SVM. Several classification performance measures are used to compare Opt-RNN-DBSVM with different classification methods and the results obtained show the good performance of the proposed method. |
| first_indexed | 2024-03-10T23:45:33Z |
| format | Article |
| id | doaj.art-5a8b442bb2364faf92667f8b99b3c323 |
| institution | Directory Open Access Journal |
| issn | 2227-7390 |
| language | English |
| last_indexed | 2024-03-10T23:45:33Z |
| publishDate | 2023-08-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Mathematics |
| spelling | doaj.art-5a8b442bb2364faf92667f8b99b3c3232023-11-19T02:03:44ZengMDPI AGMathematics2227-73902023-08-011116355510.3390/math11163555OPT-RNN-DBSVM: OPTimal Recurrent Neural Network and Density-Based Support Vector MachineKarim El Moutaouakil0Abdellatif El Ouissari1Adrian Olaru2Vasile Palade3Mihaela Ciorei4Engineering Science Laboratory, Taza Multidisciplinary Faculty, Sidi Mohamed Ben Abdellah University, Fez 30000, MoroccoEngineering Science Laboratory, Taza Multidisciplinary Faculty, Sidi Mohamed Ben Abdellah University, Fez 30000, MoroccoCentre for Computational Science and Mathematical Modelling, Coventry University, Priory Road, Coventry CV1 5FB, UKDepartment of Robotics and Production System, University Politehnica of Bucharest, 060042 Bucharest, RomaniaDepartment of Robotics and Production System, University Politehnica of Bucharest, 060042 Bucharest, RomaniaWhen implementing SVMs, two major problems are encountered: (a) the number of local minima of dual-SVM increases exponentially with the number of samples and (b) the computer storage memory required for a regular quadratic programming solver increases exponentially as the problem size expands. The Kernel-Adatron family of algorithms, gaining attention recently, has allowed us to handle very large classification and regression problems. However, these methods treat different types of samples (i.e., noise, border, and core) in the same manner, which makes these algorithms search in unpromising areas and increases the number of iterations as well. This paper introduces a hybrid method to overcome such shortcomings, called the Optimal Recurrent Neural Network and Density-Based Support Vector Machine (Opt-RNN-DBSVM). This method consists of four steps: (a) the characterization of different samples, (b) the elimination of samples with a low probability of being a support vector, (c) the construction of an appropriate recurrent neural network to solve the dual-DBSVM based on an original energy function, and (d) finding the solution to the system of differential equations that govern the dynamics of the RNN, using the Euler–Cauchy method involving an optimal time step. Density-based preprocessing reduces the number of local minima in the dual-SVM. The RNN’s recurring architecture avoids the need to explore recently visited areas. With the optimal time step, the search moves from the current vectors to the best neighboring support vectors. It is demonstrated that RNN-SVM converges to feasible support vectors and Opt-RNN-DBSVM has very low time complexity compared to the RNN-SVM with a constant time step and the Kernel-Adatron algorithm–SVM. Several classification performance measures are used to compare Opt-RNN-DBSVM with different classification methods and the results obtained show the good performance of the proposed method.https://www.mdpi.com/2227-7390/11/16/3555Recurrent Neural Network (RNN)Density-Based AlgorithmSupport Vector Machine (SVM)Kernel-Adatron algorithm (KA)Euler–Cauchy algorithm |
| spellingShingle | Karim El Moutaouakil Abdellatif El Ouissari Adrian Olaru Vasile Palade Mihaela Ciorei OPT-RNN-DBSVM: OPTimal Recurrent Neural Network and Density-Based Support Vector Machine Mathematics Recurrent Neural Network (RNN) Density-Based Algorithm Support Vector Machine (SVM) Kernel-Adatron algorithm (KA) Euler–Cauchy algorithm |
| title | OPT-RNN-DBSVM: OPTimal Recurrent Neural Network and Density-Based Support Vector Machine |
| title_full | OPT-RNN-DBSVM: OPTimal Recurrent Neural Network and Density-Based Support Vector Machine |
| title_fullStr | OPT-RNN-DBSVM: OPTimal Recurrent Neural Network and Density-Based Support Vector Machine |
| title_full_unstemmed | OPT-RNN-DBSVM: OPTimal Recurrent Neural Network and Density-Based Support Vector Machine |
| title_short | OPT-RNN-DBSVM: OPTimal Recurrent Neural Network and Density-Based Support Vector Machine |
| title_sort | opt rnn dbsvm optimal recurrent neural network and density based support vector machine |
| topic | Recurrent Neural Network (RNN) Density-Based Algorithm Support Vector Machine (SVM) Kernel-Adatron algorithm (KA) Euler–Cauchy algorithm |
| url | https://www.mdpi.com/2227-7390/11/16/3555 |
| work_keys_str_mv | AT karimelmoutaouakil optrnndbsvmoptimalrecurrentneuralnetworkanddensitybasedsupportvectormachine AT abdellatifelouissari optrnndbsvmoptimalrecurrentneuralnetworkanddensitybasedsupportvectormachine AT adrianolaru optrnndbsvmoptimalrecurrentneuralnetworkanddensitybasedsupportvectormachine AT vasilepalade optrnndbsvmoptimalrecurrentneuralnetworkanddensitybasedsupportvectormachine AT mihaelaciorei optrnndbsvmoptimalrecurrentneuralnetworkanddensitybasedsupportvectormachine |