An Automated Recognition of Work Activity in Industrial Manufacturing Using Convolutional Neural Networks
The automated assessment and analysis of employee activity in a manufacturing enterprise, operating in accordance with the concept of Industry 4.0, is essential for a quick and precise diagnosis of work quality, especially in the process of training a new employee. In the case of industrial solution...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2021-11-01
|
| Series: | Electronics |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2079-9292/10/23/2946 |
| _version_ | 1797507955232342016 |
|---|---|
| author | Justyna Patalas-Maliszewska Daniel Halikowski Robertas Damaševičius |
| author_facet | Justyna Patalas-Maliszewska Daniel Halikowski Robertas Damaševičius |
| author_sort | Justyna Patalas-Maliszewska |
| collection | DOAJ |
| description | The automated assessment and analysis of employee activity in a manufacturing enterprise, operating in accordance with the concept of Industry 4.0, is essential for a quick and precise diagnosis of work quality, especially in the process of training a new employee. In the case of industrial solutions, many approaches involving the recognition and detection of work activity are based on Convolutional Neural Networks (CNNs). Despite the wide use of CNNs, it is difficult to find solutions supporting the automated checking of work activities performed by trained employees. We propose a novel framework for the automatic generation of workplace instructions and real-time recognition of worker activities. The proposed method integrates CNN, CNN Support Vector Machine (SVM), CNN Region-Based CNN (Yolov3 Tiny) for recognizing and checking the completed work tasks. First, video recordings of the work process are analyzed and reference video frames corresponding to work activity stages are determined. Next, work-related features and objects are determined using CNN with SVM (achieving 94% accuracy) and Yolov3 Tiny network based on the characteristics of the reference frames. Additionally, matching matrix between the reference frames and the test frames using mean absolute error (MAE) as a measure of errors between paired observations was built. Finally, the practical usefulness of the proposed approach by applying the method for supporting the automatic training of new employees and checking the correctness of their work done on solid fuel boiler equipment in a manufacturing company was demonstrated. The developed information system can be integrated with other Industry 4.0 technologies introduced within an enterprise. |
| first_indexed | 2024-03-10T04:55:46Z |
| format | Article |
| id | doaj.art-e81ed04d806f46048786daf21235c3c0 |
| institution | Directory Open Access Journal |
| issn | 2079-9292 |
| language | English |
| last_indexed | 2024-03-10T04:55:46Z |
| publishDate | 2021-11-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Electronics |
| spelling | doaj.art-e81ed04d806f46048786daf21235c3c02023-11-23T02:16:31ZengMDPI AGElectronics2079-92922021-11-011023294610.3390/electronics10232946An Automated Recognition of Work Activity in Industrial Manufacturing Using Convolutional Neural NetworksJustyna Patalas-Maliszewska0Daniel Halikowski1Robertas Damaševičius2Institute of Mechanical Engineering, University of Zielona Góra, 65-417 Zielona Góra, PolandFaculty of Technical Science, University of Applied Science in Nysa, 48-300 Nysa, PolandDepartment of Applied Informatics, Vytautas Magnus University, 44404 Kaunas, LithuaniaThe automated assessment and analysis of employee activity in a manufacturing enterprise, operating in accordance with the concept of Industry 4.0, is essential for a quick and precise diagnosis of work quality, especially in the process of training a new employee. In the case of industrial solutions, many approaches involving the recognition and detection of work activity are based on Convolutional Neural Networks (CNNs). Despite the wide use of CNNs, it is difficult to find solutions supporting the automated checking of work activities performed by trained employees. We propose a novel framework for the automatic generation of workplace instructions and real-time recognition of worker activities. The proposed method integrates CNN, CNN Support Vector Machine (SVM), CNN Region-Based CNN (Yolov3 Tiny) for recognizing and checking the completed work tasks. First, video recordings of the work process are analyzed and reference video frames corresponding to work activity stages are determined. Next, work-related features and objects are determined using CNN with SVM (achieving 94% accuracy) and Yolov3 Tiny network based on the characteristics of the reference frames. Additionally, matching matrix between the reference frames and the test frames using mean absolute error (MAE) as a measure of errors between paired observations was built. Finally, the practical usefulness of the proposed approach by applying the method for supporting the automatic training of new employees and checking the correctness of their work done on solid fuel boiler equipment in a manufacturing company was demonstrated. The developed information system can be integrated with other Industry 4.0 technologies introduced within an enterprise.https://www.mdpi.com/2079-9292/10/23/2946employee work recognitionemployee training supporthuman resources managementbusiness intelligencedeep learningIndustry 4.0 |
| spellingShingle | Justyna Patalas-Maliszewska Daniel Halikowski Robertas Damaševičius An Automated Recognition of Work Activity in Industrial Manufacturing Using Convolutional Neural Networks Electronics employee work recognition employee training support human resources management business intelligence deep learning Industry 4.0 |
| title | An Automated Recognition of Work Activity in Industrial Manufacturing Using Convolutional Neural Networks |
| title_full | An Automated Recognition of Work Activity in Industrial Manufacturing Using Convolutional Neural Networks |
| title_fullStr | An Automated Recognition of Work Activity in Industrial Manufacturing Using Convolutional Neural Networks |
| title_full_unstemmed | An Automated Recognition of Work Activity in Industrial Manufacturing Using Convolutional Neural Networks |
| title_short | An Automated Recognition of Work Activity in Industrial Manufacturing Using Convolutional Neural Networks |
| title_sort | automated recognition of work activity in industrial manufacturing using convolutional neural networks |
| topic | employee work recognition employee training support human resources management business intelligence deep learning Industry 4.0 |
| url | https://www.mdpi.com/2079-9292/10/23/2946 |
| work_keys_str_mv | AT justynapatalasmaliszewska anautomatedrecognitionofworkactivityinindustrialmanufacturingusingconvolutionalneuralnetworks AT danielhalikowski anautomatedrecognitionofworkactivityinindustrialmanufacturingusingconvolutionalneuralnetworks AT robertasdamasevicius anautomatedrecognitionofworkactivityinindustrialmanufacturingusingconvolutionalneuralnetworks AT justynapatalasmaliszewska automatedrecognitionofworkactivityinindustrialmanufacturingusingconvolutionalneuralnetworks AT danielhalikowski automatedrecognitionofworkactivityinindustrialmanufacturingusingconvolutionalneuralnetworks AT robertasdamasevicius automatedrecognitionofworkactivityinindustrialmanufacturingusingconvolutionalneuralnetworks |