Variable Selection and Regularization in Quantile Regression via Minimum Covariance Determinant Based Weights
The importance of variable selection and regularization procedures in multiple regression analysis cannot be overemphasized. These procedures are adversely affected by predictor space data aberrations as well as outliers in the response space. To counter the latter, robust statistical procedures suc...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2020-12-01
|
| Series: | Entropy |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1099-4300/23/1/33 |
| _version_ | 1797543294458134528 |
|---|---|
| author | Edmore Ranganai Innocent Mudhombo |
| author_facet | Edmore Ranganai Innocent Mudhombo |
| author_sort | Edmore Ranganai |
| collection | DOAJ |
| description | The importance of variable selection and regularization procedures in multiple regression analysis cannot be overemphasized. These procedures are adversely affected by predictor space data aberrations as well as outliers in the response space. To counter the latter, robust statistical procedures such as quantile regression which generalizes the well-known least absolute deviation procedure to all quantile levels have been proposed in the literature. Quantile regression is robust to response variable outliers but very susceptible to outliers in the predictor space (high leverage points) which may alter the eigen-structure of the predictor matrix. High leverage points that alter the eigen-structure of the predictor matrix by creating or hiding collinearity are referred to as collinearity influential points. In this paper, we suggest generalizing the penalized weighted least absolute deviation to all quantile levels, i.e., to penalized weighted quantile regression using the RIDGE, LASSO, and elastic net penalties as a remedy against collinearity influential points and high leverage points in general. To maintain robustness, we make use of very robust weights based on the computationally intensive high breakdown minimum covariance determinant. Simulations and applications to well-known data sets from the literature show an improvement in variable selection and regularization due to the robust weighting formulation. |
| first_indexed | 2024-03-10T13:42:31Z |
| format | Article |
| id | doaj.art-4c6e409b045c4bb8a1eadbe9b54a33f1 |
| institution | Directory Open Access Journal |
| issn | 1099-4300 |
| language | English |
| last_indexed | 2024-03-10T13:42:31Z |
| publishDate | 2020-12-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Entropy |
| spelling | doaj.art-4c6e409b045c4bb8a1eadbe9b54a33f12023-11-21T02:53:21ZengMDPI AGEntropy1099-43002020-12-012313310.3390/e23010033Variable Selection and Regularization in Quantile Regression via Minimum Covariance Determinant Based WeightsEdmore Ranganai0Innocent Mudhombo1Department of Statistics, University of South Africa, Florida Campus, Private Bag X6, Florida Park, Roodepoort 1710, South AfricaDepartment of Accountancy, Vaal University of Technology, Vanderbijlpark Campus, Vanderbijlpark 1900, South AfricaThe importance of variable selection and regularization procedures in multiple regression analysis cannot be overemphasized. These procedures are adversely affected by predictor space data aberrations as well as outliers in the response space. To counter the latter, robust statistical procedures such as quantile regression which generalizes the well-known least absolute deviation procedure to all quantile levels have been proposed in the literature. Quantile regression is robust to response variable outliers but very susceptible to outliers in the predictor space (high leverage points) which may alter the eigen-structure of the predictor matrix. High leverage points that alter the eigen-structure of the predictor matrix by creating or hiding collinearity are referred to as collinearity influential points. In this paper, we suggest generalizing the penalized weighted least absolute deviation to all quantile levels, i.e., to penalized weighted quantile regression using the RIDGE, LASSO, and elastic net penalties as a remedy against collinearity influential points and high leverage points in general. To maintain robustness, we make use of very robust weights based on the computationally intensive high breakdown minimum covariance determinant. Simulations and applications to well-known data sets from the literature show an improvement in variable selection and regularization due to the robust weighting formulation.https://www.mdpi.com/1099-4300/23/1/33weighted quantile regressionRIDGE penaltyLASSO penaltyelastic net penaltyhigh leverage pointscollinearity influential points |
| spellingShingle | Edmore Ranganai Innocent Mudhombo Variable Selection and Regularization in Quantile Regression via Minimum Covariance Determinant Based Weights Entropy weighted quantile regression RIDGE penalty LASSO penalty elastic net penalty high leverage points collinearity influential points |
| title | Variable Selection and Regularization in Quantile Regression via Minimum Covariance Determinant Based Weights |
| title_full | Variable Selection and Regularization in Quantile Regression via Minimum Covariance Determinant Based Weights |
| title_fullStr | Variable Selection and Regularization in Quantile Regression via Minimum Covariance Determinant Based Weights |
| title_full_unstemmed | Variable Selection and Regularization in Quantile Regression via Minimum Covariance Determinant Based Weights |
| title_short | Variable Selection and Regularization in Quantile Regression via Minimum Covariance Determinant Based Weights |
| title_sort | variable selection and regularization in quantile regression via minimum covariance determinant based weights |
| topic | weighted quantile regression RIDGE penalty LASSO penalty elastic net penalty high leverage points collinearity influential points |
| url | https://www.mdpi.com/1099-4300/23/1/33 |
| work_keys_str_mv | AT edmoreranganai variableselectionandregularizationinquantileregressionviaminimumcovariancedeterminantbasedweights AT innocentmudhombo variableselectionandregularizationinquantileregressionviaminimumcovariancedeterminantbasedweights |