A Deep Neural Network Regularization Measure: The Class-Based Decorrelation Method
In response to the challenge of overfitting, which may lead to a decline in network generalization performance, this paper proposes a new regularization technique, called the class-based decorrelation method (CDM). Specifically, this method views the neurons in a specific hidden layer as base learne...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-12-01
|
| Series: | Entropy |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1099-4300/26/1/7 |
| _version_ | 1827372373646508032 |
|---|---|
| author | Chenguang Zhang Tian Liu Xuejiao Du |
| author_facet | Chenguang Zhang Tian Liu Xuejiao Du |
| author_sort | Chenguang Zhang |
| collection | DOAJ |
| description | In response to the challenge of overfitting, which may lead to a decline in network generalization performance, this paper proposes a new regularization technique, called the class-based decorrelation method (CDM). Specifically, this method views the neurons in a specific hidden layer as base learners, and aims to boost network generalization as well as model accuracy by minimizing the correlation among individual base learners while simultaneously maximizing their class-conditional correlation. Intuitively, CDM not only promotes diversity among the hidden neurons, but also enhances their cohesiveness among them when processing samples from the same class. Comparative experiments conducted on various datasets using deep models demonstrate that CDM effectively reduces overfitting and improves classification performance. |
| first_indexed | 2024-03-08T10:57:42Z |
| format | Article |
| id | doaj.art-dec702a9baa24eb4ac128ba3f64f2570 |
| institution | Directory Open Access Journal |
| issn | 1099-4300 |
| language | English |
| last_indexed | 2024-03-08T10:57:42Z |
| publishDate | 2023-12-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Entropy |
| spelling | doaj.art-dec702a9baa24eb4ac128ba3f64f25702024-01-26T16:22:44ZengMDPI AGEntropy1099-43002023-12-01261710.3390/e26010007A Deep Neural Network Regularization Measure: The Class-Based Decorrelation MethodChenguang Zhang0Tian Liu1Xuejiao Du2School of Mathematics and Statistics, Hainan University, Haikou 570100, ChinaSchool of Information and Communication Engineering, Hainan University, Haikou 570100, ChinaSchool of Mathematics and Statistics, Hainan University, Haikou 570100, ChinaIn response to the challenge of overfitting, which may lead to a decline in network generalization performance, this paper proposes a new regularization technique, called the class-based decorrelation method (CDM). Specifically, this method views the neurons in a specific hidden layer as base learners, and aims to boost network generalization as well as model accuracy by minimizing the correlation among individual base learners while simultaneously maximizing their class-conditional correlation. Intuitively, CDM not only promotes diversity among the hidden neurons, but also enhances their cohesiveness among them when processing samples from the same class. Comparative experiments conducted on various datasets using deep models demonstrate that CDM effectively reduces overfitting and improves classification performance.https://www.mdpi.com/1099-4300/26/1/7deep neural networkgeneralization abilityregularization method |
| spellingShingle | Chenguang Zhang Tian Liu Xuejiao Du A Deep Neural Network Regularization Measure: The Class-Based Decorrelation Method Entropy deep neural network generalization ability regularization method |
| title | A Deep Neural Network Regularization Measure: The Class-Based Decorrelation Method |
| title_full | A Deep Neural Network Regularization Measure: The Class-Based Decorrelation Method |
| title_fullStr | A Deep Neural Network Regularization Measure: The Class-Based Decorrelation Method |
| title_full_unstemmed | A Deep Neural Network Regularization Measure: The Class-Based Decorrelation Method |
| title_short | A Deep Neural Network Regularization Measure: The Class-Based Decorrelation Method |
| title_sort | deep neural network regularization measure the class based decorrelation method |
| topic | deep neural network generalization ability regularization method |
| url | https://www.mdpi.com/1099-4300/26/1/7 |
| work_keys_str_mv | AT chenguangzhang adeepneuralnetworkregularizationmeasuretheclassbaseddecorrelationmethod AT tianliu adeepneuralnetworkregularizationmeasuretheclassbaseddecorrelationmethod AT xuejiaodu adeepneuralnetworkregularizationmeasuretheclassbaseddecorrelationmethod AT chenguangzhang deepneuralnetworkregularizationmeasuretheclassbaseddecorrelationmethod AT tianliu deepneuralnetworkregularizationmeasuretheclassbaseddecorrelationmethod AT xuejiaodu deepneuralnetworkregularizationmeasuretheclassbaseddecorrelationmethod |