A Progressive Deep Neural Network Training Method for Image Classification with Noisy Labels
Deep neural networks (DNNs) require large amounts of labeled data for model training. However, label noise is a common problem in datasets due to the difficulty of classification and high cost of labeling processes. Introducing the concepts of curriculum learning and progressive learning, this paper...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2022-12-01
|
| Series: | Applied Sciences |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2076-3417/12/24/12754 |
| _version_ | 1797461641809362944 |
|---|---|
| author | Xuguo Yan Xuhui Xia Lei Wang Zelin Zhang |
| author_facet | Xuguo Yan Xuhui Xia Lei Wang Zelin Zhang |
| author_sort | Xuguo Yan |
| collection | DOAJ |
| description | Deep neural networks (DNNs) require large amounts of labeled data for model training. However, label noise is a common problem in datasets due to the difficulty of classification and high cost of labeling processes. Introducing the concepts of curriculum learning and progressive learning, this paper presents a novel solution that is able to handle massive noisy labels and improve model generalization ability. It proposes a new network model training strategy that considers mislabeled samples directly in the network training process. The new learning curriculum is designed to measures the complexity of the data with their distribution density in a feature space. The sample data in each category are then divided into easy-to-classify (clean samples), relatively easy-to-classify, and hard-to-classify (noisy samples) subsets according to the smallest intra-class local density with each cluster. On this basis, DNNs are trained progressively in three stages, from easy to hard, i.e., from clean to noisy samples. The experimental results demonstrate that the accuracy of image classification can be improved through data augmentation, and the classification accuracy of the proposed method is clearly higher than that of standard Inception_v2 for the NEU dataset after data augmentation, when the proportion of noisy labels in the training set does not exceed 60%. With 50% noisy labels in the training set, the classification accuracy of the proposed method outperformed recent state-of-the-art label noise learning methods, CleanNet and MentorNet. The proposed method also performed well in practical applications, where the number of noisy labels was uncertain and unevenly distributed. In this case, the proposed method not only can alleviate the adverse effects of noisy labels, but it can also improve the generalization ability of standard deep networks and their overall capability. |
| first_indexed | 2024-03-09T17:22:19Z |
| format | Article |
| id | doaj.art-a18089f78061431ea3d22bceb93d9899 |
| institution | Directory Open Access Journal |
| issn | 2076-3417 |
| language | English |
| last_indexed | 2024-03-09T17:22:19Z |
| publishDate | 2022-12-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Applied Sciences |
| spelling | doaj.art-a18089f78061431ea3d22bceb93d98992023-11-24T13:04:11ZengMDPI AGApplied Sciences2076-34172022-12-0112241275410.3390/app122412754A Progressive Deep Neural Network Training Method for Image Classification with Noisy LabelsXuguo Yan0Xuhui Xia1Lei Wang2Zelin Zhang3Key Laboratory of Metallurgical Equipment and Control Technology, Ministry of Education, Wuhan University of Science and Technology, Wuhan 430081, ChinaKey Laboratory of Metallurgical Equipment and Control Technology, Ministry of Education, Wuhan University of Science and Technology, Wuhan 430081, ChinaKey Laboratory of Metallurgical Equipment and Control Technology, Ministry of Education, Wuhan University of Science and Technology, Wuhan 430081, ChinaKey Laboratory of Metallurgical Equipment and Control Technology, Ministry of Education, Wuhan University of Science and Technology, Wuhan 430081, ChinaDeep neural networks (DNNs) require large amounts of labeled data for model training. However, label noise is a common problem in datasets due to the difficulty of classification and high cost of labeling processes. Introducing the concepts of curriculum learning and progressive learning, this paper presents a novel solution that is able to handle massive noisy labels and improve model generalization ability. It proposes a new network model training strategy that considers mislabeled samples directly in the network training process. The new learning curriculum is designed to measures the complexity of the data with their distribution density in a feature space. The sample data in each category are then divided into easy-to-classify (clean samples), relatively easy-to-classify, and hard-to-classify (noisy samples) subsets according to the smallest intra-class local density with each cluster. On this basis, DNNs are trained progressively in three stages, from easy to hard, i.e., from clean to noisy samples. The experimental results demonstrate that the accuracy of image classification can be improved through data augmentation, and the classification accuracy of the proposed method is clearly higher than that of standard Inception_v2 for the NEU dataset after data augmentation, when the proportion of noisy labels in the training set does not exceed 60%. With 50% noisy labels in the training set, the classification accuracy of the proposed method outperformed recent state-of-the-art label noise learning methods, CleanNet and MentorNet. The proposed method also performed well in practical applications, where the number of noisy labels was uncertain and unevenly distributed. In this case, the proposed method not only can alleviate the adverse effects of noisy labels, but it can also improve the generalization ability of standard deep networks and their overall capability.https://www.mdpi.com/2076-3417/12/24/12754DNNscurriculum learningprogressive learningnoisy labelsimage classification |
| spellingShingle | Xuguo Yan Xuhui Xia Lei Wang Zelin Zhang A Progressive Deep Neural Network Training Method for Image Classification with Noisy Labels Applied Sciences DNNs curriculum learning progressive learning noisy labels image classification |
| title | A Progressive Deep Neural Network Training Method for Image Classification with Noisy Labels |
| title_full | A Progressive Deep Neural Network Training Method for Image Classification with Noisy Labels |
| title_fullStr | A Progressive Deep Neural Network Training Method for Image Classification with Noisy Labels |
| title_full_unstemmed | A Progressive Deep Neural Network Training Method for Image Classification with Noisy Labels |
| title_short | A Progressive Deep Neural Network Training Method for Image Classification with Noisy Labels |
| title_sort | progressive deep neural network training method for image classification with noisy labels |
| topic | DNNs curriculum learning progressive learning noisy labels image classification |
| url | https://www.mdpi.com/2076-3417/12/24/12754 |
| work_keys_str_mv | AT xuguoyan aprogressivedeepneuralnetworktrainingmethodforimageclassificationwithnoisylabels AT xuhuixia aprogressivedeepneuralnetworktrainingmethodforimageclassificationwithnoisylabels AT leiwang aprogressivedeepneuralnetworktrainingmethodforimageclassificationwithnoisylabels AT zelinzhang aprogressivedeepneuralnetworktrainingmethodforimageclassificationwithnoisylabels AT xuguoyan progressivedeepneuralnetworktrainingmethodforimageclassificationwithnoisylabels AT xuhuixia progressivedeepneuralnetworktrainingmethodforimageclassificationwithnoisylabels AT leiwang progressivedeepneuralnetworktrainingmethodforimageclassificationwithnoisylabels AT zelinzhang progressivedeepneuralnetworktrainingmethodforimageclassificationwithnoisylabels |