Gradient boosting decision tree becomes more reliable than logistic regression in predicting probability for diabetes with big data
Abstract We sought to verify the reliability of machine learning (ML) in developing diabetes prediction models by utilizing big data. To this end, we compared the reliability of gradient boosting decision tree (GBDT) and logistic regression (LR) models using data obtained from the Kokuho-database of...
| Main Authors: | , , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Nature Portfolio
2022-10-01
|
| Series: | Scientific Reports |
| Online Access: | https://doi.org/10.1038/s41598-022-20149-z |
| _version_ | 1811238784836042752 |
|---|---|
| author | Hiroe Seto Asuka Oyama Shuji Kitora Hiroshi Toki Ryohei Yamamoto Jun’ichi Kotoku Akihiro Haga Maki Shinzawa Miyae Yamakawa Sakiko Fukui Toshiki Moriyama |
| author_facet | Hiroe Seto Asuka Oyama Shuji Kitora Hiroshi Toki Ryohei Yamamoto Jun’ichi Kotoku Akihiro Haga Maki Shinzawa Miyae Yamakawa Sakiko Fukui Toshiki Moriyama |
| author_sort | Hiroe Seto |
| collection | DOAJ |
| description | Abstract We sought to verify the reliability of machine learning (ML) in developing diabetes prediction models by utilizing big data. To this end, we compared the reliability of gradient boosting decision tree (GBDT) and logistic regression (LR) models using data obtained from the Kokuho-database of the Osaka prefecture, Japan. To develop the models, we focused on 16 predictors from health checkup data from April 2013 to December 2014. A total of 277,651 eligible participants were studied. The prediction models were developed using a light gradient boosting machine (LightGBM), which is an effective GBDT implementation algorithm, and LR. Their reliabilities were measured based on expected calibration error (ECE), negative log-likelihood (Logloss), and reliability diagrams. Similarly, their classification accuracies were measured in the area under the curve (AUC). We further analyzed their reliabilities while changing the sample size for training. Among the 277,651 participants, 15,900 (7978 males and 7922 females) were newly diagnosed with diabetes within 3 years. LightGBM (LR) achieved an ECE of 0.0018 ± 0.00033 (0.0048 ± 0.00058), a Logloss of 0.167 ± 0.00062 (0.172 ± 0.00090), and an AUC of 0.844 ± 0.0025 (0.826 ± 0.0035). From sample size analysis, the reliability of LightGBM became higher than LR when the sample size increased more than $$10^4$$ 10 4 . Thus, we confirmed that GBDT provides a more reliable model than that of LR in the development of diabetes prediction models using big data. ML could potentially produce a highly reliable diabetes prediction model, a helpful tool for improving lifestyle and preventing diabetes. |
| first_indexed | 2024-04-12T12:47:13Z |
| format | Article |
| id | doaj.art-67977fc19324478f8d3f4ab3988e8482 |
| institution | Directory Open Access Journal |
| issn | 2045-2322 |
| language | English |
| last_indexed | 2024-04-12T12:47:13Z |
| publishDate | 2022-10-01 |
| publisher | Nature Portfolio |
| record_format | Article |
| series | Scientific Reports |
| spelling | doaj.art-67977fc19324478f8d3f4ab3988e84822022-12-22T03:32:34ZengNature PortfolioScientific Reports2045-23222022-10-0112111010.1038/s41598-022-20149-zGradient boosting decision tree becomes more reliable than logistic regression in predicting probability for diabetes with big dataHiroe Seto0Asuka Oyama1Shuji Kitora2Hiroshi Toki3Ryohei Yamamoto4Jun’ichi Kotoku5Akihiro Haga6Maki Shinzawa7Miyae Yamakawa8Sakiko Fukui9Toshiki Moriyama10Health Care Division, Health and Counseling Center, Osaka UniversityHealth Care Division, Health and Counseling Center, Osaka UniversityHealth Care Division, Health and Counseling Center, Osaka UniversityHealth Care Division, Health and Counseling Center, Osaka UniversityHealth Care Division, Health and Counseling Center, Osaka UniversityHealth Care Division, Health and Counseling Center, Osaka UniversityHealth Care Division, Health and Counseling Center, Osaka UniversityDepartment of Nephrology, Graduate School of Medicine, Osaka UniversityDivision of Health Sciences, Graduate School of Medicine, Osaka UniversityDivision of Health Sciences, Graduate School of Medicine, Osaka UniversityHealth Care Division, Health and Counseling Center, Osaka UniversityAbstract We sought to verify the reliability of machine learning (ML) in developing diabetes prediction models by utilizing big data. To this end, we compared the reliability of gradient boosting decision tree (GBDT) and logistic regression (LR) models using data obtained from the Kokuho-database of the Osaka prefecture, Japan. To develop the models, we focused on 16 predictors from health checkup data from April 2013 to December 2014. A total of 277,651 eligible participants were studied. The prediction models were developed using a light gradient boosting machine (LightGBM), which is an effective GBDT implementation algorithm, and LR. Their reliabilities were measured based on expected calibration error (ECE), negative log-likelihood (Logloss), and reliability diagrams. Similarly, their classification accuracies were measured in the area under the curve (AUC). We further analyzed their reliabilities while changing the sample size for training. Among the 277,651 participants, 15,900 (7978 males and 7922 females) were newly diagnosed with diabetes within 3 years. LightGBM (LR) achieved an ECE of 0.0018 ± 0.00033 (0.0048 ± 0.00058), a Logloss of 0.167 ± 0.00062 (0.172 ± 0.00090), and an AUC of 0.844 ± 0.0025 (0.826 ± 0.0035). From sample size analysis, the reliability of LightGBM became higher than LR when the sample size increased more than $$10^4$$ 10 4 . Thus, we confirmed that GBDT provides a more reliable model than that of LR in the development of diabetes prediction models using big data. ML could potentially produce a highly reliable diabetes prediction model, a helpful tool for improving lifestyle and preventing diabetes.https://doi.org/10.1038/s41598-022-20149-z |
| spellingShingle | Hiroe Seto Asuka Oyama Shuji Kitora Hiroshi Toki Ryohei Yamamoto Jun’ichi Kotoku Akihiro Haga Maki Shinzawa Miyae Yamakawa Sakiko Fukui Toshiki Moriyama Gradient boosting decision tree becomes more reliable than logistic regression in predicting probability for diabetes with big data Scientific Reports |
| title | Gradient boosting decision tree becomes more reliable than logistic regression in predicting probability for diabetes with big data |
| title_full | Gradient boosting decision tree becomes more reliable than logistic regression in predicting probability for diabetes with big data |
| title_fullStr | Gradient boosting decision tree becomes more reliable than logistic regression in predicting probability for diabetes with big data |
| title_full_unstemmed | Gradient boosting decision tree becomes more reliable than logistic regression in predicting probability for diabetes with big data |
| title_short | Gradient boosting decision tree becomes more reliable than logistic regression in predicting probability for diabetes with big data |
| title_sort | gradient boosting decision tree becomes more reliable than logistic regression in predicting probability for diabetes with big data |
| url | https://doi.org/10.1038/s41598-022-20149-z |
| work_keys_str_mv | AT hiroeseto gradientboostingdecisiontreebecomesmorereliablethanlogisticregressioninpredictingprobabilityfordiabeteswithbigdata AT asukaoyama gradientboostingdecisiontreebecomesmorereliablethanlogisticregressioninpredictingprobabilityfordiabeteswithbigdata AT shujikitora gradientboostingdecisiontreebecomesmorereliablethanlogisticregressioninpredictingprobabilityfordiabeteswithbigdata AT hiroshitoki gradientboostingdecisiontreebecomesmorereliablethanlogisticregressioninpredictingprobabilityfordiabeteswithbigdata AT ryoheiyamamoto gradientboostingdecisiontreebecomesmorereliablethanlogisticregressioninpredictingprobabilityfordiabeteswithbigdata AT junichikotoku gradientboostingdecisiontreebecomesmorereliablethanlogisticregressioninpredictingprobabilityfordiabeteswithbigdata AT akihirohaga gradientboostingdecisiontreebecomesmorereliablethanlogisticregressioninpredictingprobabilityfordiabeteswithbigdata AT makishinzawa gradientboostingdecisiontreebecomesmorereliablethanlogisticregressioninpredictingprobabilityfordiabeteswithbigdata AT miyaeyamakawa gradientboostingdecisiontreebecomesmorereliablethanlogisticregressioninpredictingprobabilityfordiabeteswithbigdata AT sakikofukui gradientboostingdecisiontreebecomesmorereliablethanlogisticregressioninpredictingprobabilityfordiabeteswithbigdata AT toshikimoriyama gradientboostingdecisiontreebecomesmorereliablethanlogisticregressioninpredictingprobabilityfordiabeteswithbigdata |