Knowledge Distillation for Enhancing a Lightweight Magnet Tile Target Detection Model: Leveraging Spatial Attention and Multi-Scale Output Features
Accurate and efficient sorting of diverse magnetic tiles during manufacturing is vital. However, challenges arise due to visual similarities among types, necessitating complex computer vision algorithms with large sizes and high computational needs. This impedes cost-effective deployment in the indu...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-11-01
|
| Series: | Electronics |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2079-9292/12/22/4589 |
| _version_ | 1797459531005952000 |
|---|---|
| author | Qinyuan Huang Kun Yang Yuzhen Zhu Long Chen Lijia Cao |
| author_facet | Qinyuan Huang Kun Yang Yuzhen Zhu Long Chen Lijia Cao |
| author_sort | Qinyuan Huang |
| collection | DOAJ |
| description | Accurate and efficient sorting of diverse magnetic tiles during manufacturing is vital. However, challenges arise due to visual similarities among types, necessitating complex computer vision algorithms with large sizes and high computational needs. This impedes cost-effective deployment in the industry, resulting in the continued use of inefficient manual sorting. To address this issue, we propose an innovative lightweight magnetic tile detection approach that improves knowledge distillation for a compressed YOLOv5s model. Incorporating spatial attention modules into different feature extraction stages of YOLOv5s during the knowledge distillation process can enhance the ability of the compressed model to learn the knowledge of intermediate feature extraction layers from the original large model at different stages. Combining different outputs to form a multi-scale output, the multi-scale output feature in the knowledge refinement process enhances the capacity of the compressed model to grasp comprehensive target knowledge in outputs. Experimental results on our self-built magnetic tile dataset demonstrate significant achievements: 0.988 mean average precision, 0.5% discrepancy compared to the teacher’s network, and an 85% model size reduction. Moreover, a 36.70% boost in inference speed is observed for single image analysis. Our method’s effectiveness is also validated by the Pascal VOC dataset results, showing potential for broader target detection scenarios. This approach offers a solution to magnetic tile target detection challenges while being expected to expand to other applications. |
| first_indexed | 2024-03-09T16:52:41Z |
| format | Article |
| id | doaj.art-553457e06b164f43a61f7595fc62059b |
| institution | Directory Open Access Journal |
| issn | 2079-9292 |
| language | English |
| last_indexed | 2024-03-09T16:52:41Z |
| publishDate | 2023-11-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Electronics |
| spelling | doaj.art-553457e06b164f43a61f7595fc62059b2023-11-24T14:39:06ZengMDPI AGElectronics2079-92922023-11-011222458910.3390/electronics12224589Knowledge Distillation for Enhancing a Lightweight Magnet Tile Target Detection Model: Leveraging Spatial Attention and Multi-Scale Output FeaturesQinyuan Huang0Kun Yang1Yuzhen Zhu2Long Chen3Lijia Cao4School of Automation and Information Engineering, Sichuan University of Science and Engineering, Zigong 643000, ChinaSchool of Automation and Information Engineering, Sichuan University of Science and Engineering, Zigong 643000, ChinaSchool of Automation and Information Engineering, Sichuan University of Science and Engineering, Zigong 643000, ChinaSchool of Automation and Information Engineering, Sichuan University of Science and Engineering, Zigong 643000, ChinaSchool of Automation and Information Engineering, Sichuan University of Science and Engineering, Zigong 643000, ChinaAccurate and efficient sorting of diverse magnetic tiles during manufacturing is vital. However, challenges arise due to visual similarities among types, necessitating complex computer vision algorithms with large sizes and high computational needs. This impedes cost-effective deployment in the industry, resulting in the continued use of inefficient manual sorting. To address this issue, we propose an innovative lightweight magnetic tile detection approach that improves knowledge distillation for a compressed YOLOv5s model. Incorporating spatial attention modules into different feature extraction stages of YOLOv5s during the knowledge distillation process can enhance the ability of the compressed model to learn the knowledge of intermediate feature extraction layers from the original large model at different stages. Combining different outputs to form a multi-scale output, the multi-scale output feature in the knowledge refinement process enhances the capacity of the compressed model to grasp comprehensive target knowledge in outputs. Experimental results on our self-built magnetic tile dataset demonstrate significant achievements: 0.988 mean average precision, 0.5% discrepancy compared to the teacher’s network, and an 85% model size reduction. Moreover, a 36.70% boost in inference speed is observed for single image analysis. Our method’s effectiveness is also validated by the Pascal VOC dataset results, showing potential for broader target detection scenarios. This approach offers a solution to magnetic tile target detection challenges while being expected to expand to other applications.https://www.mdpi.com/2079-9292/12/22/4589spatial attentionmulti-scale output featureknowledge distillationYOLOv5smagnetic tile target detection |
| spellingShingle | Qinyuan Huang Kun Yang Yuzhen Zhu Long Chen Lijia Cao Knowledge Distillation for Enhancing a Lightweight Magnet Tile Target Detection Model: Leveraging Spatial Attention and Multi-Scale Output Features Electronics spatial attention multi-scale output feature knowledge distillation YOLOv5s magnetic tile target detection |
| title | Knowledge Distillation for Enhancing a Lightweight Magnet Tile Target Detection Model: Leveraging Spatial Attention and Multi-Scale Output Features |
| title_full | Knowledge Distillation for Enhancing a Lightweight Magnet Tile Target Detection Model: Leveraging Spatial Attention and Multi-Scale Output Features |
| title_fullStr | Knowledge Distillation for Enhancing a Lightweight Magnet Tile Target Detection Model: Leveraging Spatial Attention and Multi-Scale Output Features |
| title_full_unstemmed | Knowledge Distillation for Enhancing a Lightweight Magnet Tile Target Detection Model: Leveraging Spatial Attention and Multi-Scale Output Features |
| title_short | Knowledge Distillation for Enhancing a Lightweight Magnet Tile Target Detection Model: Leveraging Spatial Attention and Multi-Scale Output Features |
| title_sort | knowledge distillation for enhancing a lightweight magnet tile target detection model leveraging spatial attention and multi scale output features |
| topic | spatial attention multi-scale output feature knowledge distillation YOLOv5s magnetic tile target detection |
| url | https://www.mdpi.com/2079-9292/12/22/4589 |
| work_keys_str_mv | AT qinyuanhuang knowledgedistillationforenhancingalightweightmagnettiletargetdetectionmodelleveragingspatialattentionandmultiscaleoutputfeatures AT kunyang knowledgedistillationforenhancingalightweightmagnettiletargetdetectionmodelleveragingspatialattentionandmultiscaleoutputfeatures AT yuzhenzhu knowledgedistillationforenhancingalightweightmagnettiletargetdetectionmodelleveragingspatialattentionandmultiscaleoutputfeatures AT longchen knowledgedistillationforenhancingalightweightmagnettiletargetdetectionmodelleveragingspatialattentionandmultiscaleoutputfeatures AT lijiacao knowledgedistillationforenhancingalightweightmagnettiletargetdetectionmodelleveragingspatialattentionandmultiscaleoutputfeatures |