Distance Correlation-Based Feature Selection in Random Forest

The Pearson correlation coefficient (<inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mi>ρ</mi></semantics></math></inline-formula>) is a commonly used measure of correlation, but it has...

Full description

Bibliographic Details
Main Authors: Suthakaran Ratnasingam, Jose Muñoz-Lopez
Format: Article
Language:English
Published: MDPI AG 2023-08-01
Series:Entropy
Subjects:
Online Access:https://www.mdpi.com/1099-4300/25/9/1250
Description
Summary:The Pearson correlation coefficient (<inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mi>ρ</mi></semantics></math></inline-formula>) is a commonly used measure of correlation, but it has limitations as it only measures the linear relationship between two numerical variables. The distance correlation measures all types of dependencies between random vectors <i>X</i> and <i>Y</i> in arbitrary dimensions, not just the linear ones. In this paper, we propose a filter method that utilizes distance correlation as a criterion for feature selection in Random Forest regression. We conduct extensive simulation studies to evaluate its performance compared to existing methods under various data settings, in terms of the prediction mean squared error. The results show that our proposed method is competitive with existing methods and outperforms all other methods in high-dimensional (<inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mrow><mi>p</mi><mo>≥</mo><mn>300</mn></mrow></semantics></math></inline-formula>) nonlinearly related data sets. The applicability of the proposed method is also illustrated by two real data applications.
ISSN:1099-4300