Information Theoretic Multi-Target Feature Selection via Output Space Quantization
A key challenge in information theoretic feature selection is to estimate mutual information expressions that capture three desirable terms—the relevancy of a feature with the output, the redundancy and the complementarity between groups of features. The challenge becomes more pronounced i...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2019-08-01
|
| Series: | Entropy |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1099-4300/21/9/855 |
| _version_ | 1818036898258485248 |
|---|---|
| author | Konstantinos Sechidis Eleftherios Spyromitros-Xioufis Ioannis Vlahavas |
| author_facet | Konstantinos Sechidis Eleftherios Spyromitros-Xioufis Ioannis Vlahavas |
| author_sort | Konstantinos Sechidis |
| collection | DOAJ |
| description | A key challenge in information theoretic feature selection is to estimate mutual information expressions that capture three desirable terms—the relevancy of a feature with the output, the redundancy and the complementarity between groups of features. The challenge becomes more pronounced in multi-target problems, where the output space is multi-dimensional. Our work presents an algorithm that captures these three desirable terms and is suitable for the well-known multi-target prediction settings of multi-label/dimensional classification and multivariate regression. We achieve this by combining two ideas—deriving low-order information theoretic approximations for the input space and using quantization algorithms for deriving low-dimensional approximations of the output space. Under the above framework we derive a novel criterion, <i>G</i>roup-JMI-Rand, which captures various high-order target interactions. In an extensive experimental study we showed that our suggested criterion achieves competing performance against various other information theoretic feature selection criteria suggested in the literature. |
| first_indexed | 2024-12-10T07:18:16Z |
| format | Article |
| id | doaj.art-98f447350bee4e178ff79c6e4c64e9b8 |
| institution | Directory Open Access Journal |
| issn | 1099-4300 |
| language | English |
| last_indexed | 2024-12-10T07:18:16Z |
| publishDate | 2019-08-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Entropy |
| spelling | doaj.art-98f447350bee4e178ff79c6e4c64e9b82022-12-22T01:57:53ZengMDPI AGEntropy1099-43002019-08-0121985510.3390/e21090855e21090855Information Theoretic Multi-Target Feature Selection via Output Space QuantizationKonstantinos Sechidis0Eleftherios Spyromitros-Xioufis1Ioannis Vlahavas2Department of Computer Science, Aristotle University, 54124 Thessaloniki, GreeceDepartment of Computer Science, Aristotle University, 54124 Thessaloniki, GreeceDepartment of Computer Science, Aristotle University, 54124 Thessaloniki, GreeceA key challenge in information theoretic feature selection is to estimate mutual information expressions that capture three desirable terms—the relevancy of a feature with the output, the redundancy and the complementarity between groups of features. The challenge becomes more pronounced in multi-target problems, where the output space is multi-dimensional. Our work presents an algorithm that captures these three desirable terms and is suitable for the well-known multi-target prediction settings of multi-label/dimensional classification and multivariate regression. We achieve this by combining two ideas—deriving low-order information theoretic approximations for the input space and using quantization algorithms for deriving low-dimensional approximations of the output space. Under the above framework we derive a novel criterion, <i>G</i>roup-JMI-Rand, which captures various high-order target interactions. In an extensive experimental study we showed that our suggested criterion achieves competing performance against various other information theoretic feature selection criteria suggested in the literature.https://www.mdpi.com/1099-4300/21/9/855feature selectionmutual informationmulti-targetmulti-labelclustering |
| spellingShingle | Konstantinos Sechidis Eleftherios Spyromitros-Xioufis Ioannis Vlahavas Information Theoretic Multi-Target Feature Selection via Output Space Quantization Entropy feature selection mutual information multi-target multi-label clustering |
| title | Information Theoretic Multi-Target Feature Selection via Output Space Quantization |
| title_full | Information Theoretic Multi-Target Feature Selection via Output Space Quantization |
| title_fullStr | Information Theoretic Multi-Target Feature Selection via Output Space Quantization |
| title_full_unstemmed | Information Theoretic Multi-Target Feature Selection via Output Space Quantization |
| title_short | Information Theoretic Multi-Target Feature Selection via Output Space Quantization |
| title_sort | information theoretic multi target feature selection via output space quantization |
| topic | feature selection mutual information multi-target multi-label clustering |
| url | https://www.mdpi.com/1099-4300/21/9/855 |
| work_keys_str_mv | AT konstantinossechidis informationtheoreticmultitargetfeatureselectionviaoutputspacequantization AT eleftheriosspyromitrosxioufis informationtheoreticmultitargetfeatureselectionviaoutputspacequantization AT ioannisvlahavas informationtheoreticmultitargetfeatureselectionviaoutputspacequantization |