Improving accuracy metric with precision and recall metrics for optimizing stochastic classifier
All stochastic classifiers attempt to improve their classification performance by constructing an optimized classifier. Typically, all of stochastic classification algorithms employ accuracy metric to discriminate an optimal solution. However, the use of accuracy metric could lead the solution towar...
| Main Authors: | , , , |
|---|---|
| Format: | Conference or Workshop Item |
| Language: | English |
| Published: |
Universiti Utara Malaysia Press
2011
|
| Online Access: | http://psasir.upm.edu.my/id/eprint/59131/1/105.pdf |
| _version_ | 1825931944693596160 |
|---|---|
| author | Hossin, Mohammad Sulaiman, Md. Nasir Mustapha, Norwati O. K. Rahmat, Rahmita Wirza |
| author_facet | Hossin, Mohammad Sulaiman, Md. Nasir Mustapha, Norwati O. K. Rahmat, Rahmita Wirza |
| author_sort | Hossin, Mohammad |
| collection | UPM |
| description | All stochastic classifiers attempt to improve their classification performance by constructing an optimized classifier. Typically, all of stochastic classification algorithms employ accuracy metric to discriminate an optimal solution. However, the use of accuracy metric could lead the solution towards the sub-optimal solution due less discriminating power. Moreover, the accuracy metric also unable to perform optimally when dealing with imbalanced class distribution. In this study, we propose a new evaluation metric that combines accuracy metric with the extended precision and recall metrics to negate these detrimental effects. We refer the new evaluation metric as optimized accuracy with recall-precision (OARP). This paper demonstrates that the OARP metric is more discriminating than the accuracy metric and able to perform optimally when dealing with imbalanced class distribution using one simple counter-example. We also demonstrate empirically that a naïve stochastic classification algorithm, which is Monte Carlo Sampling (MCS) algorithm trained with the OARP metric, is able to obtain better predictive results than the one trained with the accuracy and F-Measure metrics. Additionally, the t-test analysis also shows a clear advantage of the MCS model trained with the OARP metric over the two selected metrics for almost five medical data sets. |
| first_indexed | 2024-03-06T09:34:26Z |
| format | Conference or Workshop Item |
| id | upm.eprints-59131 |
| institution | Universiti Putra Malaysia |
| language | English |
| last_indexed | 2024-03-06T09:34:26Z |
| publishDate | 2011 |
| publisher | Universiti Utara Malaysia Press |
| record_format | dspace |
| spelling | upm.eprints-591312018-02-22T08:51:25Z http://psasir.upm.edu.my/id/eprint/59131/ Improving accuracy metric with precision and recall metrics for optimizing stochastic classifier Hossin, Mohammad Sulaiman, Md. Nasir Mustapha, Norwati O. K. Rahmat, Rahmita Wirza All stochastic classifiers attempt to improve their classification performance by constructing an optimized classifier. Typically, all of stochastic classification algorithms employ accuracy metric to discriminate an optimal solution. However, the use of accuracy metric could lead the solution towards the sub-optimal solution due less discriminating power. Moreover, the accuracy metric also unable to perform optimally when dealing with imbalanced class distribution. In this study, we propose a new evaluation metric that combines accuracy metric with the extended precision and recall metrics to negate these detrimental effects. We refer the new evaluation metric as optimized accuracy with recall-precision (OARP). This paper demonstrates that the OARP metric is more discriminating than the accuracy metric and able to perform optimally when dealing with imbalanced class distribution using one simple counter-example. We also demonstrate empirically that a naïve stochastic classification algorithm, which is Monte Carlo Sampling (MCS) algorithm trained with the OARP metric, is able to obtain better predictive results than the one trained with the accuracy and F-Measure metrics. Additionally, the t-test analysis also shows a clear advantage of the MCS model trained with the OARP metric over the two selected metrics for almost five medical data sets. Universiti Utara Malaysia Press 2011 Conference or Workshop Item PeerReviewed text en http://psasir.upm.edu.my/id/eprint/59131/1/105.pdf Hossin, Mohammad and Sulaiman, Md. Nasir and Mustapha, Norwati and O. K. Rahmat, Rahmita Wirza (2011) Improving accuracy metric with precision and recall metrics for optimizing stochastic classifier. In: 3rd International Conference on Computing and Informatics (ICOCI 2011), 8-9 June 2011, Bandung, Indonesia. (pp. 105-110). |
| spellingShingle | Hossin, Mohammad Sulaiman, Md. Nasir Mustapha, Norwati O. K. Rahmat, Rahmita Wirza Improving accuracy metric with precision and recall metrics for optimizing stochastic classifier |
| title | Improving accuracy metric with precision and recall metrics for optimizing stochastic classifier |
| title_full | Improving accuracy metric with precision and recall metrics for optimizing stochastic classifier |
| title_fullStr | Improving accuracy metric with precision and recall metrics for optimizing stochastic classifier |
| title_full_unstemmed | Improving accuracy metric with precision and recall metrics for optimizing stochastic classifier |
| title_short | Improving accuracy metric with precision and recall metrics for optimizing stochastic classifier |
| title_sort | improving accuracy metric with precision and recall metrics for optimizing stochastic classifier |
| url | http://psasir.upm.edu.my/id/eprint/59131/1/105.pdf |
| work_keys_str_mv | AT hossinmohammad improvingaccuracymetricwithprecisionandrecallmetricsforoptimizingstochasticclassifier AT sulaimanmdnasir improvingaccuracymetricwithprecisionandrecallmetricsforoptimizingstochasticclassifier AT mustaphanorwati improvingaccuracymetricwithprecisionandrecallmetricsforoptimizingstochasticclassifier AT okrahmatrahmitawirza improvingaccuracymetricwithprecisionandrecallmetricsforoptimizingstochasticclassifier |