CoDC: Accurate Learning with Noisy Labels via Disagreement and Consistency
Inspired by the biological nervous system, deep neural networks (DNNs) are able to achieve remarkable performance in various tasks. However, they struggle to handle label noise, which can poison the memorization effects of DNNs. Co-teaching-based methods are popular in learning with noisy labels. Th...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2024-02-01
|
| Series: | Biomimetics |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2313-7673/9/2/92 |
| _version_ | 1797298833875533824 |
|---|---|
| author | Yongfeng Dong Jiawei Li Zhen Wang Wenyu Jia |
| author_facet | Yongfeng Dong Jiawei Li Zhen Wang Wenyu Jia |
| author_sort | Yongfeng Dong |
| collection | DOAJ |
| description | Inspired by the biological nervous system, deep neural networks (DNNs) are able to achieve remarkable performance in various tasks. However, they struggle to handle label noise, which can poison the memorization effects of DNNs. Co-teaching-based methods are popular in learning with noisy labels. These methods cross-train two DNNs based on the small-loss criterion and employ a strategy using either “disagreement” or “consistency” to obtain the divergence of the two networks. However, these methods are sample-inefficient for generalization in noisy scenarios. In this paper, we propose CoDC, a novel <b>C</b>o-teaching-basedmethod for accurate learning with label noise via both <b>D</b>isagreement and <b>C</b>onsistency strategies. Specifically, CoDC maintains disagreement at the feature level and consistency at the prediction level using a balanced loss function. Additionally, a weighted cross-entropy loss is proposed based on information derived from the historical training process. Moreover, the valuable knowledge involved in “large-loss” samples is further developed and utilized by assigning pseudo-labels. Comprehensive experiments were conducted on both synthetic and real-world noise and under various noise types. CoDC achieved 72.81% accuracy on the Clothing1M dataset and 76.96% (Top1) accuracy on the WebVision1.0 dataset. These superior results demonstrate the effectiveness and robustness of learning with noisy labels. |
| first_indexed | 2024-03-07T22:40:49Z |
| format | Article |
| id | doaj.art-8868402934e9452daa44fd99da2eda32 |
| institution | Directory Open Access Journal |
| issn | 2313-7673 |
| language | English |
| last_indexed | 2024-03-07T22:40:49Z |
| publishDate | 2024-02-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Biomimetics |
| spelling | doaj.art-8868402934e9452daa44fd99da2eda322024-02-23T15:09:06ZengMDPI AGBiomimetics2313-76732024-02-01929210.3390/biomimetics9020092CoDC: Accurate Learning with Noisy Labels via Disagreement and ConsistencyYongfeng Dong0Jiawei Li1Zhen Wang2Wenyu Jia3School of Artificial Intelligence, Hebei University of Technology, Tianjin 300401, ChinaSchool of Artificial Intelligence, Hebei University of Technology, Tianjin 300401, ChinaSchool of Artificial Intelligence, Hebei University of Technology, Tianjin 300401, ChinaSchool of Artificial Intelligence, Hebei University of Technology, Tianjin 300401, ChinaInspired by the biological nervous system, deep neural networks (DNNs) are able to achieve remarkable performance in various tasks. However, they struggle to handle label noise, which can poison the memorization effects of DNNs. Co-teaching-based methods are popular in learning with noisy labels. These methods cross-train two DNNs based on the small-loss criterion and employ a strategy using either “disagreement” or “consistency” to obtain the divergence of the two networks. However, these methods are sample-inefficient for generalization in noisy scenarios. In this paper, we propose CoDC, a novel <b>C</b>o-teaching-basedmethod for accurate learning with label noise via both <b>D</b>isagreement and <b>C</b>onsistency strategies. Specifically, CoDC maintains disagreement at the feature level and consistency at the prediction level using a balanced loss function. Additionally, a weighted cross-entropy loss is proposed based on information derived from the historical training process. Moreover, the valuable knowledge involved in “large-loss” samples is further developed and utilized by assigning pseudo-labels. Comprehensive experiments were conducted on both synthetic and real-world noise and under various noise types. CoDC achieved 72.81% accuracy on the Clothing1M dataset and 76.96% (Top1) accuracy on the WebVision1.0 dataset. These superior results demonstrate the effectiveness and robustness of learning with noisy labels.https://www.mdpi.com/2313-7673/9/2/92biological nervous systemDNNslabel noisedisagreementconsistency |
| spellingShingle | Yongfeng Dong Jiawei Li Zhen Wang Wenyu Jia CoDC: Accurate Learning with Noisy Labels via Disagreement and Consistency Biomimetics biological nervous system DNNs label noise disagreement consistency |
| title | CoDC: Accurate Learning with Noisy Labels via Disagreement and Consistency |
| title_full | CoDC: Accurate Learning with Noisy Labels via Disagreement and Consistency |
| title_fullStr | CoDC: Accurate Learning with Noisy Labels via Disagreement and Consistency |
| title_full_unstemmed | CoDC: Accurate Learning with Noisy Labels via Disagreement and Consistency |
| title_short | CoDC: Accurate Learning with Noisy Labels via Disagreement and Consistency |
| title_sort | codc accurate learning with noisy labels via disagreement and consistency |
| topic | biological nervous system DNNs label noise disagreement consistency |
| url | https://www.mdpi.com/2313-7673/9/2/92 |
| work_keys_str_mv | AT yongfengdong codcaccuratelearningwithnoisylabelsviadisagreementandconsistency AT jiaweili codcaccuratelearningwithnoisylabelsviadisagreementandconsistency AT zhenwang codcaccuratelearningwithnoisylabelsviadisagreementandconsistency AT wenyujia codcaccuratelearningwithnoisylabelsviadisagreementandconsistency |