Medical prediction from missing data with max-minus negative regularized dropout
Missing data is a naturally common problem faced in medical research. Imputation is a widely used technique to alleviate this problem. Unfortunately, the inherent uncertainty of imputation would make the model overfit the observed data distribution, which has a negative impact on the model generaliz...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Frontiers Media S.A.
2023-07-01
|
| Series: | Frontiers in Neuroscience |
| Subjects: | |
| Online Access: | https://www.frontiersin.org/articles/10.3389/fnins.2023.1221970/full |
| _version_ | 1797780961258110976 |
|---|---|
| author | Lvhui Hu Xiaoen Cheng Chuanbiao Wen Yulan Ren |
| author_facet | Lvhui Hu Xiaoen Cheng Chuanbiao Wen Yulan Ren |
| author_sort | Lvhui Hu |
| collection | DOAJ |
| description | Missing data is a naturally common problem faced in medical research. Imputation is a widely used technique to alleviate this problem. Unfortunately, the inherent uncertainty of imputation would make the model overfit the observed data distribution, which has a negative impact on the model generalization performance. R-Drop is a powerful technique to regularize the training of deep neural networks. However, it fails to differentiate the positive and negative samples, which prevents the model from learning robust representations. To handle this problem, we propose a novel negative regularization enhanced R-Drop scheme to boost performance and generalization ability, particularly in the context of missing data. The negative regularization enhanced R-Drop additionally forces the output distributions of positive and negative samples to be inconsistent with each other. Especially, we design a new max-minus negative sampling technique that uses the maximum in-batch values to minus the mini-batch to yield the negative samples to provide sufficient diversity for the model. We test the resulting max-minus negative regularized dropout method on three real-world medical prediction datasets, including both missing and complete cases, to show the effectiveness of the proposed method. |
| first_indexed | 2024-03-12T23:51:10Z |
| format | Article |
| id | doaj.art-55aaf59e0e1a4eb39309d43c9688dfbc |
| institution | Directory Open Access Journal |
| issn | 1662-453X |
| language | English |
| last_indexed | 2024-03-12T23:51:10Z |
| publishDate | 2023-07-01 |
| publisher | Frontiers Media S.A. |
| record_format | Article |
| series | Frontiers in Neuroscience |
| spelling | doaj.art-55aaf59e0e1a4eb39309d43c9688dfbc2023-07-13T13:59:33ZengFrontiers Media S.A.Frontiers in Neuroscience1662-453X2023-07-011710.3389/fnins.2023.12219701221970Medical prediction from missing data with max-minus negative regularized dropoutLvhui Hu0Xiaoen Cheng1Chuanbiao Wen2Yulan Ren3School of Intelligent Medicine, Chengdu University of Traditional Chinese Medicine, Chengdu, ChinaSchool of Intelligent Medicine, Chengdu University of Traditional Chinese Medicine, Chengdu, ChinaSchool of Intelligent Medicine, Chengdu University of Traditional Chinese Medicine, Chengdu, ChinaSinology College of Chengdu University of Traditional Chinese Medicine, Chengdu, ChinaMissing data is a naturally common problem faced in medical research. Imputation is a widely used technique to alleviate this problem. Unfortunately, the inherent uncertainty of imputation would make the model overfit the observed data distribution, which has a negative impact on the model generalization performance. R-Drop is a powerful technique to regularize the training of deep neural networks. However, it fails to differentiate the positive and negative samples, which prevents the model from learning robust representations. To handle this problem, we propose a novel negative regularization enhanced R-Drop scheme to boost performance and generalization ability, particularly in the context of missing data. The negative regularization enhanced R-Drop additionally forces the output distributions of positive and negative samples to be inconsistent with each other. Especially, we design a new max-minus negative sampling technique that uses the maximum in-batch values to minus the mini-batch to yield the negative samples to provide sufficient diversity for the model. We test the resulting max-minus negative regularized dropout method on three real-world medical prediction datasets, including both missing and complete cases, to show the effectiveness of the proposed method.https://www.frontiersin.org/articles/10.3389/fnins.2023.1221970/fullmedical predictionmissing datadropoutregularizationnegative sampling |
| spellingShingle | Lvhui Hu Xiaoen Cheng Chuanbiao Wen Yulan Ren Medical prediction from missing data with max-minus negative regularized dropout Frontiers in Neuroscience medical prediction missing data dropout regularization negative sampling |
| title | Medical prediction from missing data with max-minus negative regularized dropout |
| title_full | Medical prediction from missing data with max-minus negative regularized dropout |
| title_fullStr | Medical prediction from missing data with max-minus negative regularized dropout |
| title_full_unstemmed | Medical prediction from missing data with max-minus negative regularized dropout |
| title_short | Medical prediction from missing data with max-minus negative regularized dropout |
| title_sort | medical prediction from missing data with max minus negative regularized dropout |
| topic | medical prediction missing data dropout regularization negative sampling |
| url | https://www.frontiersin.org/articles/10.3389/fnins.2023.1221970/full |
| work_keys_str_mv | AT lvhuihu medicalpredictionfrommissingdatawithmaxminusnegativeregularizeddropout AT xiaoencheng medicalpredictionfrommissingdatawithmaxminusnegativeregularizeddropout AT chuanbiaowen medicalpredictionfrommissingdatawithmaxminusnegativeregularizeddropout AT yulanren medicalpredictionfrommissingdatawithmaxminusnegativeregularizeddropout |