Reverse Knowledge Distillation with Two Teachers for Industrial Defect Detection
Industrial defect detection plays an important role in smart manufacturing and is widely used in various scenarios such as smart inspection and product quality control. Currently, although utilizing a framework for knowledge distillation to identify industrial defects has achieved great progress, it...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-03-01
|
| Series: | Applied Sciences |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2076-3417/13/6/3838 |
| _version_ | 1797613610779803648 |
|---|---|
| author | Mingjing Pei Ningzhong Liu Pan Gao Han Sun |
| author_facet | Mingjing Pei Ningzhong Liu Pan Gao Han Sun |
| author_sort | Mingjing Pei |
| collection | DOAJ |
| description | Industrial defect detection plays an important role in smart manufacturing and is widely used in various scenarios such as smart inspection and product quality control. Currently, although utilizing a framework for knowledge distillation to identify industrial defects has achieved great progress, it is still a significant challenge task to extract better image features and prevent overfitting for student networks. In this study, a reverse knowledge distillation framework with two teachers is designed. First, for the teacher network, two teachers with different architectures are used to extract the diverse features of the images from multiple models. Second, considering the different contributions of channels and different teacher networks, the attention mechanism and iterative attention feature fusion idea are introduced. Finally, to prevent overfitting, the student network is designed with a network architecture that is inconsistent with the teacher network. Extensive experiments were conducted on Mvtec and BTAD datasets, which are industrial defect detection datasets. On the Mvtec dataset, the average accuracy values of image-level and pixel-level ROC achieved 99.43% and 97.87%, respectively. On the BTAD dataset, the average accuracy values of image-level and pixel-level ROC reached 94% and 98%, respectively. The performance on both datasets is significantly improved, demonstrating the effectiveness of our method. |
| first_indexed | 2024-03-11T06:58:16Z |
| format | Article |
| id | doaj.art-1cfa0439dab141a980831c87427942ee |
| institution | Directory Open Access Journal |
| issn | 2076-3417 |
| language | English |
| last_indexed | 2024-03-11T06:58:16Z |
| publishDate | 2023-03-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Applied Sciences |
| spelling | doaj.art-1cfa0439dab141a980831c87427942ee2023-11-17T09:27:39ZengMDPI AGApplied Sciences2076-34172023-03-01136383810.3390/app13063838Reverse Knowledge Distillation with Two Teachers for Industrial Defect DetectionMingjing Pei0Ningzhong Liu1Pan Gao2Han Sun3College of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics, Nanjing 211106, ChinaCollege of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics, Nanjing 211106, ChinaCollege of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics, Nanjing 211106, ChinaCollege of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics, Nanjing 211106, ChinaIndustrial defect detection plays an important role in smart manufacturing and is widely used in various scenarios such as smart inspection and product quality control. Currently, although utilizing a framework for knowledge distillation to identify industrial defects has achieved great progress, it is still a significant challenge task to extract better image features and prevent overfitting for student networks. In this study, a reverse knowledge distillation framework with two teachers is designed. First, for the teacher network, two teachers with different architectures are used to extract the diverse features of the images from multiple models. Second, considering the different contributions of channels and different teacher networks, the attention mechanism and iterative attention feature fusion idea are introduced. Finally, to prevent overfitting, the student network is designed with a network architecture that is inconsistent with the teacher network. Extensive experiments were conducted on Mvtec and BTAD datasets, which are industrial defect detection datasets. On the Mvtec dataset, the average accuracy values of image-level and pixel-level ROC achieved 99.43% and 97.87%, respectively. On the BTAD dataset, the average accuracy values of image-level and pixel-level ROC reached 94% and 98%, respectively. The performance on both datasets is significantly improved, demonstrating the effectiveness of our method.https://www.mdpi.com/2076-3417/13/6/3838knowledge distillationindustrial defect detectionanomaly detectiondeep learning |
| spellingShingle | Mingjing Pei Ningzhong Liu Pan Gao Han Sun Reverse Knowledge Distillation with Two Teachers for Industrial Defect Detection Applied Sciences knowledge distillation industrial defect detection anomaly detection deep learning |
| title | Reverse Knowledge Distillation with Two Teachers for Industrial Defect Detection |
| title_full | Reverse Knowledge Distillation with Two Teachers for Industrial Defect Detection |
| title_fullStr | Reverse Knowledge Distillation with Two Teachers for Industrial Defect Detection |
| title_full_unstemmed | Reverse Knowledge Distillation with Two Teachers for Industrial Defect Detection |
| title_short | Reverse Knowledge Distillation with Two Teachers for Industrial Defect Detection |
| title_sort | reverse knowledge distillation with two teachers for industrial defect detection |
| topic | knowledge distillation industrial defect detection anomaly detection deep learning |
| url | https://www.mdpi.com/2076-3417/13/6/3838 |
| work_keys_str_mv | AT mingjingpei reverseknowledgedistillationwithtwoteachersforindustrialdefectdetection AT ningzhongliu reverseknowledgedistillationwithtwoteachersforindustrialdefectdetection AT pangao reverseknowledgedistillationwithtwoteachersforindustrialdefectdetection AT hansun reverseknowledgedistillationwithtwoteachersforindustrialdefectdetection |