A hybrid evaluation metric for optimizing classifier
The accuracy metric has been widely used for discriminating and selecting an optimal solution in constructing an optimized classifier. However, the use of accuracy metric leads the searching process to the sub-optimal solutions due to its limited capability of discriminating values. In this study, w...
| Main Authors: | , , , , |
|---|---|
| Format: | Conference or Workshop Item |
| Language: | English |
| Published: |
IEEE
2011
|
| Online Access: | http://psasir.upm.edu.my/id/eprint/68284/1/A%20hybrid%20evaluation%20metric%20for%20optimizing%20classifier.pdf |
| _version_ | 1825933496187617280 |
|---|---|
| author | Hossin, Mohammad Sulaiman, Md. Nasir Mustapha, Aida Mustapha, Norwati O. K. Rahmat, Rahmita Wirza |
| author_facet | Hossin, Mohammad Sulaiman, Md. Nasir Mustapha, Aida Mustapha, Norwati O. K. Rahmat, Rahmita Wirza |
| author_sort | Hossin, Mohammad |
| collection | UPM |
| description | The accuracy metric has been widely used for discriminating and selecting an optimal solution in constructing an optimized classifier. However, the use of accuracy metric leads the searching process to the sub-optimal solutions due to its limited capability of discriminating values. In this study, we propose a hybrid evaluation metric, which combines the accuracy metric with the precision and recall metrics. We call this new performance metric as Optimized Accuracy with Recall-Precision (OARP). This paper demonstrates that the OARP metric is more discriminating than the accuracy metric using two counter-examples. To verify this advantage, we conduct an empirical verification using a statistical discriminative analysis to prove that the OARP is statistically more discriminating than the accuracy metric. We also empirically demonstrate that a naive stochastic classification algorithm trained with the OARP metric is able to obtain better predictive results than the one trained with the conventional accuracy metric. The experiments have proved that the OARP metric is a better evaluator and optimizer in the constructing of optimized classifier. |
| first_indexed | 2024-03-06T09:58:10Z |
| format | Conference or Workshop Item |
| id | upm.eprints-68284 |
| institution | Universiti Putra Malaysia |
| language | English |
| last_indexed | 2024-03-06T09:58:10Z |
| publishDate | 2011 |
| publisher | IEEE |
| record_format | dspace |
| spelling | upm.eprints-682842019-05-10T08:29:47Z http://psasir.upm.edu.my/id/eprint/68284/ A hybrid evaluation metric for optimizing classifier Hossin, Mohammad Sulaiman, Md. Nasir Mustapha, Aida Mustapha, Norwati O. K. Rahmat, Rahmita Wirza The accuracy metric has been widely used for discriminating and selecting an optimal solution in constructing an optimized classifier. However, the use of accuracy metric leads the searching process to the sub-optimal solutions due to its limited capability of discriminating values. In this study, we propose a hybrid evaluation metric, which combines the accuracy metric with the precision and recall metrics. We call this new performance metric as Optimized Accuracy with Recall-Precision (OARP). This paper demonstrates that the OARP metric is more discriminating than the accuracy metric using two counter-examples. To verify this advantage, we conduct an empirical verification using a statistical discriminative analysis to prove that the OARP is statistically more discriminating than the accuracy metric. We also empirically demonstrate that a naive stochastic classification algorithm trained with the OARP metric is able to obtain better predictive results than the one trained with the conventional accuracy metric. The experiments have proved that the OARP metric is a better evaluator and optimizer in the constructing of optimized classifier. IEEE 2011 Conference or Workshop Item PeerReviewed text en http://psasir.upm.edu.my/id/eprint/68284/1/A%20hybrid%20evaluation%20metric%20for%20optimizing%20classifier.pdf Hossin, Mohammad and Sulaiman, Md. Nasir and Mustapha, Aida and Mustapha, Norwati and O. K. Rahmat, Rahmita Wirza (2011) A hybrid evaluation metric for optimizing classifier. In: 2011 3rd Conference on Data Mining and Optimization (DMO), 28-29 June 2011, Putrajaya, Malaysia. (pp. 165-170). 10.1109/DMO.2011.5976522 |
| spellingShingle | Hossin, Mohammad Sulaiman, Md. Nasir Mustapha, Aida Mustapha, Norwati O. K. Rahmat, Rahmita Wirza A hybrid evaluation metric for optimizing classifier |
| title | A hybrid evaluation metric for optimizing classifier |
| title_full | A hybrid evaluation metric for optimizing classifier |
| title_fullStr | A hybrid evaluation metric for optimizing classifier |
| title_full_unstemmed | A hybrid evaluation metric for optimizing classifier |
| title_short | A hybrid evaluation metric for optimizing classifier |
| title_sort | hybrid evaluation metric for optimizing classifier |
| url | http://psasir.upm.edu.my/id/eprint/68284/1/A%20hybrid%20evaluation%20metric%20for%20optimizing%20classifier.pdf |
| work_keys_str_mv | AT hossinmohammad ahybridevaluationmetricforoptimizingclassifier AT sulaimanmdnasir ahybridevaluationmetricforoptimizingclassifier AT mustaphaaida ahybridevaluationmetricforoptimizingclassifier AT mustaphanorwati ahybridevaluationmetricforoptimizingclassifier AT okrahmatrahmitawirza ahybridevaluationmetricforoptimizingclassifier AT hossinmohammad hybridevaluationmetricforoptimizingclassifier AT sulaimanmdnasir hybridevaluationmetricforoptimizingclassifier AT mustaphaaida hybridevaluationmetricforoptimizingclassifier AT mustaphanorwati hybridevaluationmetricforoptimizingclassifier AT okrahmatrahmitawirza hybridevaluationmetricforoptimizingclassifier |