A Low-Cost Detail-Aware Neural Network Framework and Its Application in Mask Wearing Monitoring
The use of deep learning techniques in real-time monitoring can save a lot of manpower in various scenarios. For example, mask-wearing is an effective measure to prevent COVID-19 and other respiratory diseases, especially for vulnerable populations such as children, the elderly, and people with unde...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-08-01
|
| Series: | Applied Sciences |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2076-3417/13/17/9747 |
| _version_ | 1797582811112144896 |
|---|---|
| author | Silei Cao Shun Long Fangting Liao |
| author_facet | Silei Cao Shun Long Fangting Liao |
| author_sort | Silei Cao |
| collection | DOAJ |
| description | The use of deep learning techniques in real-time monitoring can save a lot of manpower in various scenarios. For example, mask-wearing is an effective measure to prevent COVID-19 and other respiratory diseases, especially for vulnerable populations such as children, the elderly, and people with underlying health problems. Currently, many public places such as hospitals, nursing homes, social service facilities, and schools experiencing outbreaks require mandatory mask-wearing. However, most of the terminal devices currently available have very limited GPU capability to run large neural networks. This means that we have to keep the parameter size of a neural network modest while maintaining its performance. In this paper, we propose a framework that applies deep learning techniques to real-time monitoring and uses it for the real-time monitoring of mask-wearing status. The main contributions are as follows: First, a feature fusion technique called skip layer pooling fusion (SLPF) is proposed for image classification tasks. It fully utilizes both deep and shallow features of a convolutional neural network while minimizing the growth in model parameters caused by feature fusion. On average, this technique improves the accuracy of various neural network models by 4.78% and 5.21% on CIFAR100 and Tiny-ImageNet, respectively. Second, layer attention (LA), an attention mechanism tailor-made for feature fusion, is proposed. Since different layers of convolutional neural networks make different impacts on the final prediction results, LA learns a set of weights to better enhance the contribution of important convolutional layer features. On average, it improves the accuracy of various neural network models by 2.10% and 2.63% on CIFAR100 and Tiny-ImageNet, respectively. Third, a MobileNetv2-based lightweight mask-wearing status classification model is trained, which is suitable for deployment on mobile devices and achieves an accuracy of 95.49%. Additionally, a ResNet mask-wearing status classification model is trained, which has a larger model size but achieves high accuracy of 98.14%. By applying the proposed methods to the ResNet mask-wearing status classification model, the accuracy is improved by 1.58%. Fourth, a mask-wearing status detection model is enhanced based on YOLOv5 with a spatial-frequency fusion module resulting in a mAP improvement of 2.20%. Overall, this paper presents various techniques to improve the performance of neural networks and apply them to mask-wearing status monitoring, which can help stop pandemics. |
| first_indexed | 2024-03-10T23:27:50Z |
| format | Article |
| id | doaj.art-97b45b9e29484bdcaac80b554b768175 |
| institution | Directory Open Access Journal |
| issn | 2076-3417 |
| language | English |
| last_indexed | 2024-03-10T23:27:50Z |
| publishDate | 2023-08-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Applied Sciences |
| spelling | doaj.art-97b45b9e29484bdcaac80b554b7681752023-11-19T07:51:13ZengMDPI AGApplied Sciences2076-34172023-08-011317974710.3390/app13179747A Low-Cost Detail-Aware Neural Network Framework and Its Application in Mask Wearing MonitoringSilei Cao0Shun Long1Fangting Liao2College of Information Science and Technology, Jinan University, Guangzhou 510632, ChinaCollege of Information Science and Technology, Jinan University, Guangzhou 510632, ChinaCollege of Information Science and Technology, Jinan University, Guangzhou 510632, ChinaThe use of deep learning techniques in real-time monitoring can save a lot of manpower in various scenarios. For example, mask-wearing is an effective measure to prevent COVID-19 and other respiratory diseases, especially for vulnerable populations such as children, the elderly, and people with underlying health problems. Currently, many public places such as hospitals, nursing homes, social service facilities, and schools experiencing outbreaks require mandatory mask-wearing. However, most of the terminal devices currently available have very limited GPU capability to run large neural networks. This means that we have to keep the parameter size of a neural network modest while maintaining its performance. In this paper, we propose a framework that applies deep learning techniques to real-time monitoring and uses it for the real-time monitoring of mask-wearing status. The main contributions are as follows: First, a feature fusion technique called skip layer pooling fusion (SLPF) is proposed for image classification tasks. It fully utilizes both deep and shallow features of a convolutional neural network while minimizing the growth in model parameters caused by feature fusion. On average, this technique improves the accuracy of various neural network models by 4.78% and 5.21% on CIFAR100 and Tiny-ImageNet, respectively. Second, layer attention (LA), an attention mechanism tailor-made for feature fusion, is proposed. Since different layers of convolutional neural networks make different impacts on the final prediction results, LA learns a set of weights to better enhance the contribution of important convolutional layer features. On average, it improves the accuracy of various neural network models by 2.10% and 2.63% on CIFAR100 and Tiny-ImageNet, respectively. Third, a MobileNetv2-based lightweight mask-wearing status classification model is trained, which is suitable for deployment on mobile devices and achieves an accuracy of 95.49%. Additionally, a ResNet mask-wearing status classification model is trained, which has a larger model size but achieves high accuracy of 98.14%. By applying the proposed methods to the ResNet mask-wearing status classification model, the accuracy is improved by 1.58%. Fourth, a mask-wearing status detection model is enhanced based on YOLOv5 with a spatial-frequency fusion module resulting in a mAP improvement of 2.20%. Overall, this paper presents various techniques to improve the performance of neural networks and apply them to mask-wearing status monitoring, which can help stop pandemics.https://www.mdpi.com/2076-3417/13/17/9747mask-wearing status monitoringconvolutional neural networkglobal average poolingfeature fusionattention mechanism |
| spellingShingle | Silei Cao Shun Long Fangting Liao A Low-Cost Detail-Aware Neural Network Framework and Its Application in Mask Wearing Monitoring Applied Sciences mask-wearing status monitoring convolutional neural network global average pooling feature fusion attention mechanism |
| title | A Low-Cost Detail-Aware Neural Network Framework and Its Application in Mask Wearing Monitoring |
| title_full | A Low-Cost Detail-Aware Neural Network Framework and Its Application in Mask Wearing Monitoring |
| title_fullStr | A Low-Cost Detail-Aware Neural Network Framework and Its Application in Mask Wearing Monitoring |
| title_full_unstemmed | A Low-Cost Detail-Aware Neural Network Framework and Its Application in Mask Wearing Monitoring |
| title_short | A Low-Cost Detail-Aware Neural Network Framework and Its Application in Mask Wearing Monitoring |
| title_sort | low cost detail aware neural network framework and its application in mask wearing monitoring |
| topic | mask-wearing status monitoring convolutional neural network global average pooling feature fusion attention mechanism |
| url | https://www.mdpi.com/2076-3417/13/17/9747 |
| work_keys_str_mv | AT sileicao alowcostdetailawareneuralnetworkframeworkanditsapplicationinmaskwearingmonitoring AT shunlong alowcostdetailawareneuralnetworkframeworkanditsapplicationinmaskwearingmonitoring AT fangtingliao alowcostdetailawareneuralnetworkframeworkanditsapplicationinmaskwearingmonitoring AT sileicao lowcostdetailawareneuralnetworkframeworkanditsapplicationinmaskwearingmonitoring AT shunlong lowcostdetailawareneuralnetworkframeworkanditsapplicationinmaskwearingmonitoring AT fangtingliao lowcostdetailawareneuralnetworkframeworkanditsapplicationinmaskwearingmonitoring |