Hyper-Heuristic Evolutionary Approach for Constructing Decision Tree Classifiers
Decision tree models have earned a special status in predictive modeling since these are considered comprehensible for human analysis and insight. Classification and Regression Tree (CART) algorithm is one of the renowned decision tree induction algorithms to address the classification as well as re...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Universiti Utara Malaysia Press
2021
|
| Subjects: | |
| Online Access: | https://repo.uum.edu.my/id/eprint/28786/1/JICT%2020%2002%202021%20249-276.pdf https://doi.org/10.32890/jict2021.20.2.5 |
| _version_ | 1803629521333649408 |
|---|---|
| author | Kumar, Sunil Ratnoo, Saroj Vashishtha, Jyoti |
| author_facet | Kumar, Sunil Ratnoo, Saroj Vashishtha, Jyoti |
| author_sort | Kumar, Sunil |
| collection | UUM |
| description | Decision tree models have earned a special status in predictive modeling since these are considered comprehensible for human analysis and insight. Classification and Regression Tree (CART) algorithm is one of the renowned decision tree induction algorithms to address the classification as well as regression problems. Finding optimal values for the hyper parameters of a decision tree construction algorithm is a challenging issue. While making an effective decision tree classifier with high accuracy and comprehensibility, we need to address the question of setting optimal values for its hyper parameters like the maximum size of the tree, the minimum number of instances required in a node for inducing a split, node splitting criterion and the amount of pruning. The hyper parameter setting influences the performance of the decision tree model. As researchers, we know that no single setting of hyper parameters works equally well for different datasets. A particular setting that gives an optimal decision tree for one dataset may produce a sub-optimal decision tree model for another dataset. In this paper, we present a hyper heuristic approach for tuning the hyper parameters of Recursive and Partition Trees (rpart), which is a typical implementation of CART in statistical and data analytics package R. We employ an evolutionary algorithm as hyper heuristic for tuning the hyper parameters of the decision tree classifier. The approach is named as Hyper heuristic Evolutionary Approach with Recursive and Partition Trees (HEARpart). The proposed approach is validated on 30 datasets. It is statistically proved that HEARpart performs significantly better than WEKA's J48 algorithm in terms of error rate, F-measure, and tree size. Further, the suggested hyper heuristic algorithm constructs significantly comprehensible models as compared to WEKA's J48, CART and other similar decision tree construction strategies. The results show that the accuracy achieved by the hyper heuristic approach is slightly less as compared to the other comparative approaches. |
| first_indexed | 2024-07-04T06:39:10Z |
| format | Article |
| id | uum-28786 |
| institution | Universiti Utara Malaysia |
| language | English |
| last_indexed | 2024-07-04T06:39:10Z |
| publishDate | 2021 |
| publisher | Universiti Utara Malaysia Press |
| record_format | dspace |
| spelling | uum-287862023-06-19T15:01:29Z https://repo.uum.edu.my/id/eprint/28786/ Hyper-Heuristic Evolutionary Approach for Constructing Decision Tree Classifiers Kumar, Sunil Ratnoo, Saroj Vashishtha, Jyoti QA75 Electronic computers. Computer science Decision tree models have earned a special status in predictive modeling since these are considered comprehensible for human analysis and insight. Classification and Regression Tree (CART) algorithm is one of the renowned decision tree induction algorithms to address the classification as well as regression problems. Finding optimal values for the hyper parameters of a decision tree construction algorithm is a challenging issue. While making an effective decision tree classifier with high accuracy and comprehensibility, we need to address the question of setting optimal values for its hyper parameters like the maximum size of the tree, the minimum number of instances required in a node for inducing a split, node splitting criterion and the amount of pruning. The hyper parameter setting influences the performance of the decision tree model. As researchers, we know that no single setting of hyper parameters works equally well for different datasets. A particular setting that gives an optimal decision tree for one dataset may produce a sub-optimal decision tree model for another dataset. In this paper, we present a hyper heuristic approach for tuning the hyper parameters of Recursive and Partition Trees (rpart), which is a typical implementation of CART in statistical and data analytics package R. We employ an evolutionary algorithm as hyper heuristic for tuning the hyper parameters of the decision tree classifier. The approach is named as Hyper heuristic Evolutionary Approach with Recursive and Partition Trees (HEARpart). The proposed approach is validated on 30 datasets. It is statistically proved that HEARpart performs significantly better than WEKA's J48 algorithm in terms of error rate, F-measure, and tree size. Further, the suggested hyper heuristic algorithm constructs significantly comprehensible models as compared to WEKA's J48, CART and other similar decision tree construction strategies. The results show that the accuracy achieved by the hyper heuristic approach is slightly less as compared to the other comparative approaches. Universiti Utara Malaysia Press 2021 Article PeerReviewed application/pdf en cc4_by https://repo.uum.edu.my/id/eprint/28786/1/JICT%2020%2002%202021%20249-276.pdf Kumar, Sunil and Ratnoo, Saroj and Vashishtha, Jyoti (2021) Hyper-Heuristic Evolutionary Approach for Constructing Decision Tree Classifiers. Journal of Information and Communication Technology, 20 (02). pp. 249-276. ISSN 2180-3862 https://e-journal.uum.edu.my/index.php/jict/article/view/jict2021.20.2.5 https://doi.org/10.32890/jict2021.20.2.5 https://doi.org/10.32890/jict2021.20.2.5 |
| spellingShingle | QA75 Electronic computers. Computer science Kumar, Sunil Ratnoo, Saroj Vashishtha, Jyoti Hyper-Heuristic Evolutionary Approach for Constructing Decision Tree Classifiers |
| title | Hyper-Heuristic Evolutionary Approach for Constructing Decision Tree Classifiers |
| title_full | Hyper-Heuristic Evolutionary Approach for Constructing Decision Tree Classifiers |
| title_fullStr | Hyper-Heuristic Evolutionary Approach for Constructing Decision Tree Classifiers |
| title_full_unstemmed | Hyper-Heuristic Evolutionary Approach for Constructing Decision Tree Classifiers |
| title_short | Hyper-Heuristic Evolutionary Approach for Constructing Decision Tree Classifiers |
| title_sort | hyper heuristic evolutionary approach for constructing decision tree classifiers |
| topic | QA75 Electronic computers. Computer science |
| url | https://repo.uum.edu.my/id/eprint/28786/1/JICT%2020%2002%202021%20249-276.pdf https://doi.org/10.32890/jict2021.20.2.5 |
| work_keys_str_mv | AT kumarsunil hyperheuristicevolutionaryapproachforconstructingdecisiontreeclassifiers AT ratnoosaroj hyperheuristicevolutionaryapproachforconstructingdecisiontreeclassifiers AT vashishthajyoti hyperheuristicevolutionaryapproachforconstructingdecisiontreeclassifiers |