IMMIGRATE: A Margin-based Feature Selection Method with Interaction Terms
Traditional hypothesis-margin researches focus on obtaining large margins and feature selection. In this work, we show that the robustness of margins is also critical and can be measured using entropy. In addition, our approach provides clear mathematical formulations and explanations to uncover fea...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2020-03-01
|
| Series: | Entropy |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1099-4300/22/3/291 |
| _version_ | 1811303308798722048 |
|---|---|
| author | Ruzhang Zhao Pengyu Hong Jun S. Liu |
| author_facet | Ruzhang Zhao Pengyu Hong Jun S. Liu |
| author_sort | Ruzhang Zhao |
| collection | DOAJ |
| description | Traditional hypothesis-margin researches focus on obtaining large margins and feature selection. In this work, we show that the robustness of margins is also critical and can be measured using entropy. In addition, our approach provides clear mathematical formulations and explanations to uncover feature interactions, which is often lack in large hypothesis-margin based approaches. We design an algorithm, termed IMMIGRATE (Iterative max-min entropy margin-maximization with interaction terms), for training the weights associated with the interaction terms. IMMIGRATE simultaneously utilizes both local and global information and can be used as a base learner in Boosting. We evaluate IMMIGRATE in a wide range of tasks, in which it demonstrates exceptional robustness and achieves the state-of-the-art results with high interpretability. |
| first_indexed | 2024-04-13T07:44:48Z |
| format | Article |
| id | doaj.art-78cfc3cd33934c3ab4551dffe592a8fc |
| institution | Directory Open Access Journal |
| issn | 1099-4300 |
| language | English |
| last_indexed | 2024-04-13T07:44:48Z |
| publishDate | 2020-03-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Entropy |
| spelling | doaj.art-78cfc3cd33934c3ab4551dffe592a8fc2022-12-22T02:55:43ZengMDPI AGEntropy1099-43002020-03-0122329110.3390/e22030291e22030291IMMIGRATE: A Margin-based Feature Selection Method with Interaction TermsRuzhang Zhao0Pengyu Hong1Jun S. Liu2Department of Biostatistics, Bloomberg School of Public Health, Johns Hopkins University, Baltimore, MD 21205, USADepartment of Computer Science, Brandeis University, Waltham, MA 02453, USADepartment of Statistics, Harvard University, Cambridge, MA 02138, USATraditional hypothesis-margin researches focus on obtaining large margins and feature selection. In this work, we show that the robustness of margins is also critical and can be measured using entropy. In addition, our approach provides clear mathematical formulations and explanations to uncover feature interactions, which is often lack in large hypothesis-margin based approaches. We design an algorithm, termed IMMIGRATE (Iterative max-min entropy margin-maximization with interaction terms), for training the weights associated with the interaction terms. IMMIGRATE simultaneously utilizes both local and global information and can be used as a base learner in Boosting. We evaluate IMMIGRATE in a wide range of tasks, in which it demonstrates exceptional robustness and achieves the state-of-the-art results with high interpretability.https://www.mdpi.com/1099-4300/22/3/291hypothesis-marginfeature selectionentropyimmigrate |
| spellingShingle | Ruzhang Zhao Pengyu Hong Jun S. Liu IMMIGRATE: A Margin-based Feature Selection Method with Interaction Terms Entropy hypothesis-margin feature selection entropy immigrate |
| title | IMMIGRATE: A Margin-based Feature Selection Method with Interaction Terms |
| title_full | IMMIGRATE: A Margin-based Feature Selection Method with Interaction Terms |
| title_fullStr | IMMIGRATE: A Margin-based Feature Selection Method with Interaction Terms |
| title_full_unstemmed | IMMIGRATE: A Margin-based Feature Selection Method with Interaction Terms |
| title_short | IMMIGRATE: A Margin-based Feature Selection Method with Interaction Terms |
| title_sort | immigrate a margin based feature selection method with interaction terms |
| topic | hypothesis-margin feature selection entropy immigrate |
| url | https://www.mdpi.com/1099-4300/22/3/291 |
| work_keys_str_mv | AT ruzhangzhao immigrateamarginbasedfeatureselectionmethodwithinteractionterms AT pengyuhong immigrateamarginbasedfeatureselectionmethodwithinteractionterms AT junsliu immigrateamarginbasedfeatureselectionmethodwithinteractionterms |