Benchmarking Analysis of the Accuracy of Classification Methods Related to Entropy
In the machine learning literature we can find numerous methods to solve classification problems. We propose two new performance measures to analyze such methods. These measures are defined by using the concept of proportional reduction of classification error with respect to three benchmark classif...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2021-07-01
|
| Series: | Entropy |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1099-4300/23/7/850 |
| _version_ | 1827687573713059840 |
|---|---|
| author | Yolanda Orenes Alejandro Rabasa Jesus Javier Rodriguez-Sala Joaquin Sanchez-Soriano |
| author_facet | Yolanda Orenes Alejandro Rabasa Jesus Javier Rodriguez-Sala Joaquin Sanchez-Soriano |
| author_sort | Yolanda Orenes |
| collection | DOAJ |
| description | In the machine learning literature we can find numerous methods to solve classification problems. We propose two new performance measures to analyze such methods. These measures are defined by using the concept of proportional reduction of classification error with respect to three benchmark classifiers, the random and two intuitive classifiers which are based on how a non-expert person could realize classification simply by applying a frequentist approach. We show that these three simple methods are closely related to different aspects of the entropy of the dataset. Therefore, these measures account somewhat for entropy in the dataset when evaluating the performance of classifiers. This allows us to measure the improvement in the classification results compared to simple methods, and at the same time how entropy affects classification capacity. To illustrate how these new performance measures can be used to analyze classifiers taking into account the entropy of the dataset, we carry out an intensive experiment in which we use the well-known J48 algorithm, and a UCI repository dataset on which we have previously selected a subset of the most relevant attributes. Then we carry out an extensive experiment in which we consider four heuristic classifiers, and 11 datasets. |
| first_indexed | 2024-03-10T09:40:06Z |
| format | Article |
| id | doaj.art-4dbe36616b044ec88aa9d3265765b40e |
| institution | Directory Open Access Journal |
| issn | 1099-4300 |
| language | English |
| last_indexed | 2024-03-10T09:40:06Z |
| publishDate | 2021-07-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Entropy |
| spelling | doaj.art-4dbe36616b044ec88aa9d3265765b40e2023-11-22T03:44:45ZengMDPI AGEntropy1099-43002021-07-0123785010.3390/e23070850Benchmarking Analysis of the Accuracy of Classification Methods Related to EntropyYolanda Orenes0Alejandro Rabasa1Jesus Javier Rodriguez-Sala2Joaquin Sanchez-Soriano3I.U.I. Centro de Investigación Operativa (CIO), Universidad Miguel Hernandez de Elche, 03202 Elche, SpainI.U.I. Centro de Investigación Operativa (CIO), Universidad Miguel Hernandez de Elche, 03202 Elche, SpainI.U.I. Centro de Investigación Operativa (CIO), Universidad Miguel Hernandez de Elche, 03202 Elche, SpainI.U.I. Centro de Investigación Operativa (CIO), Universidad Miguel Hernandez de Elche, 03202 Elche, SpainIn the machine learning literature we can find numerous methods to solve classification problems. We propose two new performance measures to analyze such methods. These measures are defined by using the concept of proportional reduction of classification error with respect to three benchmark classifiers, the random and two intuitive classifiers which are based on how a non-expert person could realize classification simply by applying a frequentist approach. We show that these three simple methods are closely related to different aspects of the entropy of the dataset. Therefore, these measures account somewhat for entropy in the dataset when evaluating the performance of classifiers. This allows us to measure the improvement in the classification results compared to simple methods, and at the same time how entropy affects classification capacity. To illustrate how these new performance measures can be used to analyze classifiers taking into account the entropy of the dataset, we carry out an intensive experiment in which we use the well-known J48 algorithm, and a UCI repository dataset on which we have previously selected a subset of the most relevant attributes. Then we carry out an extensive experiment in which we consider four heuristic classifiers, and 11 datasets.https://www.mdpi.com/1099-4300/23/7/850entropyclassification methodsintuitive classification methodperformance measuresbenchmarking |
| spellingShingle | Yolanda Orenes Alejandro Rabasa Jesus Javier Rodriguez-Sala Joaquin Sanchez-Soriano Benchmarking Analysis of the Accuracy of Classification Methods Related to Entropy Entropy entropy classification methods intuitive classification method performance measures benchmarking |
| title | Benchmarking Analysis of the Accuracy of Classification Methods Related to Entropy |
| title_full | Benchmarking Analysis of the Accuracy of Classification Methods Related to Entropy |
| title_fullStr | Benchmarking Analysis of the Accuracy of Classification Methods Related to Entropy |
| title_full_unstemmed | Benchmarking Analysis of the Accuracy of Classification Methods Related to Entropy |
| title_short | Benchmarking Analysis of the Accuracy of Classification Methods Related to Entropy |
| title_sort | benchmarking analysis of the accuracy of classification methods related to entropy |
| topic | entropy classification methods intuitive classification method performance measures benchmarking |
| url | https://www.mdpi.com/1099-4300/23/7/850 |
| work_keys_str_mv | AT yolandaorenes benchmarkinganalysisoftheaccuracyofclassificationmethodsrelatedtoentropy AT alejandrorabasa benchmarkinganalysisoftheaccuracyofclassificationmethodsrelatedtoentropy AT jesusjavierrodriguezsala benchmarkinganalysisoftheaccuracyofclassificationmethodsrelatedtoentropy AT joaquinsanchezsoriano benchmarkinganalysisoftheaccuracyofclassificationmethodsrelatedtoentropy |