Oblique decision tree ensemble via twin bounded SVM
Ensemble methods with “perturb and combine” strategy have shown improved performance in the classification problems. Recently, random forest algorithm was ranked one among 179 classifiers evaluated on 121 UCI datasets. Motivated by this, we propose a new approach for the generation of oblique decisi...
| Main Authors: | , , |
|---|---|
| Other Authors: | |
| Format: | Journal Article |
| Language: | English |
| Published: |
2022
|
| Subjects: | |
| Online Access: | https://hdl.handle.net/10356/161155 |
| _version_ | 1811687608026136576 |
|---|---|
| author | Ganaie, M. A. Tanveer, M. Suganthan, Ponnuthurai Nagaratnam |
| author2 | School of Electrical and Electronic Engineering |
| author_facet | School of Electrical and Electronic Engineering Ganaie, M. A. Tanveer, M. Suganthan, Ponnuthurai Nagaratnam |
| author_sort | Ganaie, M. A. |
| collection | NTU |
| description | Ensemble methods with “perturb and combine” strategy have shown improved performance in the classification problems. Recently, random forest algorithm was ranked one among 179 classifiers evaluated on 121 UCI datasets. Motivated by this, we propose a new approach for the generation of oblique decision trees. At each non-leaf node, the training data samples are grouped in two categories based on the Bhattachrayya distance with randomly selected feature subset. Then, twin bounded support vector machine (TBSVM) is used to get two clustering hyperplanes such that each hyperplane is closer to data points of one group and as far as possible from the data points of other group. Based on these hyperplanes, each non-leaf node is splitted to generate the decision tree. In this paper, we used different base models like random forest (RaF), rotation forest (RoF), random sub rotation forest (RRoF) to generate the different oblique decision tree forests named as TBRaF, TBRoF and TBRRoF, respectively. In earlier oblique decision trees, like multisurface proximal support vector machine (MPSVM) based oblique decision trees, matrices are semi-positive definite and hence different regularization methods are required. However, no explicit regularization techniques need to be applied to the primal problems as the matrices in the proposed TBRaF, TBRoF and TBRRoF are positive definite. We evaluated the performance of the proposed models (TBRaF, TBRoF and TBRRoF) on 49 datasets taken from the UCI repository and on some real-world biological datasets (not in UCI). The experimental results and statistical tests conducted show that TBRaF and TBRRoF outperform other baseline methods. |
| first_indexed | 2024-10-01T05:19:01Z |
| format | Journal Article |
| id | ntu-10356/161155 |
| institution | Nanyang Technological University |
| language | English |
| last_indexed | 2024-10-01T05:19:01Z |
| publishDate | 2022 |
| record_format | dspace |
| spelling | ntu-10356/1611552022-08-17T00:59:22Z Oblique decision tree ensemble via twin bounded SVM Ganaie, M. A. Tanveer, M. Suganthan, Ponnuthurai Nagaratnam School of Electrical and Electronic Engineering Engineering::Electrical and electronic engineering Oblique Ensemble Ensemble methods with “perturb and combine” strategy have shown improved performance in the classification problems. Recently, random forest algorithm was ranked one among 179 classifiers evaluated on 121 UCI datasets. Motivated by this, we propose a new approach for the generation of oblique decision trees. At each non-leaf node, the training data samples are grouped in two categories based on the Bhattachrayya distance with randomly selected feature subset. Then, twin bounded support vector machine (TBSVM) is used to get two clustering hyperplanes such that each hyperplane is closer to data points of one group and as far as possible from the data points of other group. Based on these hyperplanes, each non-leaf node is splitted to generate the decision tree. In this paper, we used different base models like random forest (RaF), rotation forest (RoF), random sub rotation forest (RRoF) to generate the different oblique decision tree forests named as TBRaF, TBRoF and TBRRoF, respectively. In earlier oblique decision trees, like multisurface proximal support vector machine (MPSVM) based oblique decision trees, matrices are semi-positive definite and hence different regularization methods are required. However, no explicit regularization techniques need to be applied to the primal problems as the matrices in the proposed TBRaF, TBRoF and TBRRoF are positive definite. We evaluated the performance of the proposed models (TBRaF, TBRoF and TBRRoF) on 49 datasets taken from the UCI repository and on some real-world biological datasets (not in UCI). The experimental results and statistical tests conducted show that TBRaF and TBRRoF outperform other baseline methods. This work was supported by Science and Engineering Research Board (SERB) as Early Career Research Award grant no. ECR/2017/000053 and Department of Science and Technology as Ramanujan fellowship grant no. SB/S2/RJN-001/2016. 2022-08-17T00:59:22Z 2022-08-17T00:59:22Z 2020 Journal Article Ganaie, M. A., Tanveer, M. & Suganthan, P. N. (2020). Oblique decision tree ensemble via twin bounded SVM. Expert Systems With Applications, 143, 113072-. https://dx.doi.org/10.1016/j.eswa.2019.113072 0957-4174 https://hdl.handle.net/10356/161155 10.1016/j.eswa.2019.113072 2-s2.0-85075266172 143 113072 en Expert Systems with Applications © 2019 Elsevier Ltd. All rights reserved. |
| spellingShingle | Engineering::Electrical and electronic engineering Oblique Ensemble Ganaie, M. A. Tanveer, M. Suganthan, Ponnuthurai Nagaratnam Oblique decision tree ensemble via twin bounded SVM |
| title | Oblique decision tree ensemble via twin bounded SVM |
| title_full | Oblique decision tree ensemble via twin bounded SVM |
| title_fullStr | Oblique decision tree ensemble via twin bounded SVM |
| title_full_unstemmed | Oblique decision tree ensemble via twin bounded SVM |
| title_short | Oblique decision tree ensemble via twin bounded SVM |
| title_sort | oblique decision tree ensemble via twin bounded svm |
| topic | Engineering::Electrical and electronic engineering Oblique Ensemble |
| url | https://hdl.handle.net/10356/161155 |
| work_keys_str_mv | AT ganaiema obliquedecisiontreeensembleviatwinboundedsvm AT tanveerm obliquedecisiontreeensembleviatwinboundedsvm AT suganthanponnuthurainagaratnam obliquedecisiontreeensembleviatwinboundedsvm |