Feature Equilibrium: An Adversarial Training Method to Improve Representation Learning
Abstract Over-fitting is a significant threat to the integrity and reliability of deep neural networks with generous parameters. One problem is that the model learned more specific features than general features in the training process. To solve the problem, we propose an adversarial training method...
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Springer
2023-04-01
|
| Series: | International Journal of Computational Intelligence Systems |
| Subjects: | |
| Online Access: | https://doi.org/10.1007/s44196-023-00229-2 |
| _version_ | 1797836312137433088 |
|---|---|
| author | Minghui Liu Meiyi Yang Jiali Deng Xuan Cheng Tianshu Xie Pan Deng Haigang Gong Ming Liu Xiaomin Wang |
| author_facet | Minghui Liu Meiyi Yang Jiali Deng Xuan Cheng Tianshu Xie Pan Deng Haigang Gong Ming Liu Xiaomin Wang |
| author_sort | Minghui Liu |
| collection | DOAJ |
| description | Abstract Over-fitting is a significant threat to the integrity and reliability of deep neural networks with generous parameters. One problem is that the model learned more specific features than general features in the training process. To solve the problem, we propose an adversarial training method to assist the model in strengthening general representation learning. In this method, we make a classification model as a generator G and introduce an unsupervised discriminator D to distinguish the hidden feature of the classification model from real images to limit their spatial distance. Notably, the D will fall into the trap of a perfect discriminator resulting in the gradient of confrontation loss of 0 after overtraining. To avoid this situation, we train the D with a probability $$P_{c}$$ P c . Our proposed method is easy to incorporate into existing frameworks. It has been evaluated under various network architectures over different fields of datasets. Experiments show that this method, under low computational cost, outperforms the benchmark by 1.5–2 points on different datasets. For semantic segmentation on VOC, our proposed method achieves 2.2 points higher mAP. |
| first_indexed | 2024-04-09T15:06:48Z |
| format | Article |
| id | doaj.art-d3831f29335f41fdb62ecb765e3bfcc0 |
| institution | Directory Open Access Journal |
| issn | 1875-6883 |
| language | English |
| last_indexed | 2024-04-09T15:06:48Z |
| publishDate | 2023-04-01 |
| publisher | Springer |
| record_format | Article |
| series | International Journal of Computational Intelligence Systems |
| spelling | doaj.art-d3831f29335f41fdb62ecb765e3bfcc02023-04-30T11:27:36ZengSpringerInternational Journal of Computational Intelligence Systems1875-68832023-04-0116111210.1007/s44196-023-00229-2Feature Equilibrium: An Adversarial Training Method to Improve Representation LearningMinghui Liu0Meiyi Yang1Jiali Deng2Xuan Cheng3Tianshu Xie4Pan Deng5Haigang Gong6Ming Liu7Xiaomin Wang8Yangzte Delta Region Institute(Quzhou), University of Electronic Science and Technology of ChinaSchool of Computer Science and Engineering, University of Electronic Science and Technology of ChinaSchool of Computer Science and Engineering, University of Electronic Science and Technology of ChinaSchool of Computer Science and Engineering, University of Electronic Science and Technology of ChinaSchool of Computer Science and Engineering, University of Electronic Science and Technology of ChinaSchool of Computer Science and Engineering, University of Electronic Science and Technology of ChinaSchool of Computer Science and Engineering, University of Electronic Science and Technology of ChinaSchool of Computer Science and Engineering, University of Electronic Science and Technology of ChinaThe Quzhou Affiliated Hospital, Wenzhou Medical UniversityAbstract Over-fitting is a significant threat to the integrity and reliability of deep neural networks with generous parameters. One problem is that the model learned more specific features than general features in the training process. To solve the problem, we propose an adversarial training method to assist the model in strengthening general representation learning. In this method, we make a classification model as a generator G and introduce an unsupervised discriminator D to distinguish the hidden feature of the classification model from real images to limit their spatial distance. Notably, the D will fall into the trap of a perfect discriminator resulting in the gradient of confrontation loss of 0 after overtraining. To avoid this situation, we train the D with a probability $$P_{c}$$ P c . Our proposed method is easy to incorporate into existing frameworks. It has been evaluated under various network architectures over different fields of datasets. Experiments show that this method, under low computational cost, outperforms the benchmark by 1.5–2 points on different datasets. For semantic segmentation on VOC, our proposed method achieves 2.2 points higher mAP.https://doi.org/10.1007/s44196-023-00229-2Over-fittingRepresentation learningAdversarial trainingUnsupervised discriminator |
| spellingShingle | Minghui Liu Meiyi Yang Jiali Deng Xuan Cheng Tianshu Xie Pan Deng Haigang Gong Ming Liu Xiaomin Wang Feature Equilibrium: An Adversarial Training Method to Improve Representation Learning International Journal of Computational Intelligence Systems Over-fitting Representation learning Adversarial training Unsupervised discriminator |
| title | Feature Equilibrium: An Adversarial Training Method to Improve Representation Learning |
| title_full | Feature Equilibrium: An Adversarial Training Method to Improve Representation Learning |
| title_fullStr | Feature Equilibrium: An Adversarial Training Method to Improve Representation Learning |
| title_full_unstemmed | Feature Equilibrium: An Adversarial Training Method to Improve Representation Learning |
| title_short | Feature Equilibrium: An Adversarial Training Method to Improve Representation Learning |
| title_sort | feature equilibrium an adversarial training method to improve representation learning |
| topic | Over-fitting Representation learning Adversarial training Unsupervised discriminator |
| url | https://doi.org/10.1007/s44196-023-00229-2 |
| work_keys_str_mv | AT minghuiliu featureequilibriumanadversarialtrainingmethodtoimproverepresentationlearning AT meiyiyang featureequilibriumanadversarialtrainingmethodtoimproverepresentationlearning AT jialideng featureequilibriumanadversarialtrainingmethodtoimproverepresentationlearning AT xuancheng featureequilibriumanadversarialtrainingmethodtoimproverepresentationlearning AT tianshuxie featureequilibriumanadversarialtrainingmethodtoimproverepresentationlearning AT pandeng featureequilibriumanadversarialtrainingmethodtoimproverepresentationlearning AT haiganggong featureequilibriumanadversarialtrainingmethodtoimproverepresentationlearning AT mingliu featureequilibriumanadversarialtrainingmethodtoimproverepresentationlearning AT xiaominwang featureequilibriumanadversarialtrainingmethodtoimproverepresentationlearning |