A Double-Penalized Estimator to Combat Separation and Multicollinearity in Logistic Regression
When developing prediction models for small or sparse binary data with many highly correlated covariates, logistic regression often encounters separation or multicollinearity problems, resulting serious bias and even the nonexistence of standard maximum likelihood estimates. The combination of separ...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2022-10-01
|
| Series: | Mathematics |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2227-7390/10/20/3824 |
| _version_ | 1797471706386792448 |
|---|---|
| author | Ying Guan Guang-Hui Fu |
| author_facet | Ying Guan Guang-Hui Fu |
| author_sort | Ying Guan |
| collection | DOAJ |
| description | When developing prediction models for small or sparse binary data with many highly correlated covariates, logistic regression often encounters separation or multicollinearity problems, resulting serious bias and even the nonexistence of standard maximum likelihood estimates. The combination of separation and multicollinearity makes the task of logistic regression more difficult, and a few studies addressed separation and multicollinearity simultaneously. In this paper, we propose a double-penalized method called lFRE to combat separation and multicollinearity in logistic regression. lFRE combines the log<i>F</i>-type penalty with the ridge penalty. The results indicate that compared with other penalty methods, lFRE can not only effectively remove bias from predicted probabilities but also provide the minimum mean squared prediction error. Aside from that, a real dataset is also employed to test the performance of the lFRE algorithm compared with several existing methods. The result shows that lFRE has strong competitiveness compared with them and can be used as an alternative algorithm in logistic regression to solve separation and multicollinearity problems. |
| first_indexed | 2024-03-09T19:52:02Z |
| format | Article |
| id | doaj.art-3a01a37e50c44e8c93755ff1e56a1eb4 |
| institution | Directory Open Access Journal |
| issn | 2227-7390 |
| language | English |
| last_indexed | 2024-03-09T19:52:02Z |
| publishDate | 2022-10-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Mathematics |
| spelling | doaj.art-3a01a37e50c44e8c93755ff1e56a1eb42023-11-24T01:07:31ZengMDPI AGMathematics2227-73902022-10-011020382410.3390/math10203824A Double-Penalized Estimator to Combat Separation and Multicollinearity in Logistic RegressionYing Guan0Guang-Hui Fu1School of Science, Kunming University of Science and Technology, Kunming 650500, ChinaSchool of Science, Kunming University of Science and Technology, Kunming 650500, ChinaWhen developing prediction models for small or sparse binary data with many highly correlated covariates, logistic regression often encounters separation or multicollinearity problems, resulting serious bias and even the nonexistence of standard maximum likelihood estimates. The combination of separation and multicollinearity makes the task of logistic regression more difficult, and a few studies addressed separation and multicollinearity simultaneously. In this paper, we propose a double-penalized method called lFRE to combat separation and multicollinearity in logistic regression. lFRE combines the log<i>F</i>-type penalty with the ridge penalty. The results indicate that compared with other penalty methods, lFRE can not only effectively remove bias from predicted probabilities but also provide the minimum mean squared prediction error. Aside from that, a real dataset is also employed to test the performance of the lFRE algorithm compared with several existing methods. The result shows that lFRE has strong competitiveness compared with them and can be used as an alternative algorithm in logistic regression to solve separation and multicollinearity problems.https://www.mdpi.com/2227-7390/10/20/3824double-penalized likelihood estimatorlog<i>F</i>-type penaltylogistic regressionmulticollinearityseparation |
| spellingShingle | Ying Guan Guang-Hui Fu A Double-Penalized Estimator to Combat Separation and Multicollinearity in Logistic Regression Mathematics double-penalized likelihood estimator log<i>F</i>-type penalty logistic regression multicollinearity separation |
| title | A Double-Penalized Estimator to Combat Separation and Multicollinearity in Logistic Regression |
| title_full | A Double-Penalized Estimator to Combat Separation and Multicollinearity in Logistic Regression |
| title_fullStr | A Double-Penalized Estimator to Combat Separation and Multicollinearity in Logistic Regression |
| title_full_unstemmed | A Double-Penalized Estimator to Combat Separation and Multicollinearity in Logistic Regression |
| title_short | A Double-Penalized Estimator to Combat Separation and Multicollinearity in Logistic Regression |
| title_sort | double penalized estimator to combat separation and multicollinearity in logistic regression |
| topic | double-penalized likelihood estimator log<i>F</i>-type penalty logistic regression multicollinearity separation |
| url | https://www.mdpi.com/2227-7390/10/20/3824 |
| work_keys_str_mv | AT yingguan adoublepenalizedestimatortocombatseparationandmulticollinearityinlogisticregression AT guanghuifu adoublepenalizedestimatortocombatseparationandmulticollinearityinlogisticregression AT yingguan doublepenalizedestimatortocombatseparationandmulticollinearityinlogisticregression AT guanghuifu doublepenalizedestimatortocombatseparationandmulticollinearityinlogisticregression |