Vision-based Human Presence Detection by Means of Transfer Learning Approach
Human–robot interaction (HRI) and human–robot collaboration (HRC) has become more important as the industries are stepping towards the phase of digitalization and Industry 4.0. Indeed, the emphasis is often placed on the safety of the physical well-being of the human workers. To safeguard the human...
| Main Authors: | , , , , , |
|---|---|
| Format: | Book Chapter |
| Language: | English |
| Published: |
Springer, Singapore
2022
|
| Subjects: | |
| Online Access: | http://umpir.ump.edu.my/id/eprint/37059/1/Vision-Based%20Human%20Presence%20Detection%20by%20Means%20of%20Transfer%20Learning%20Approach.pdf |
| _version_ | 1796995609765347328 |
|---|---|
| author | Tang, Jin Cheng Ahmad Fakhri, Ab. Nasir Anwar, P. P. Abdul Majeed Mohd Azraai, Mohd Razman Ismail, Mohd Khairuddin Thai, Li Lim |
| author_facet | Tang, Jin Cheng Ahmad Fakhri, Ab. Nasir Anwar, P. P. Abdul Majeed Mohd Azraai, Mohd Razman Ismail, Mohd Khairuddin Thai, Li Lim |
| author_sort | Tang, Jin Cheng |
| collection | UMP |
| description | Human–robot interaction (HRI) and human–robot collaboration (HRC) has become more important as the industries are stepping towards the phase of digitalization and Industry 4.0. Indeed, the emphasis is often placed on the safety of the physical well-being of the human workers. To safeguard the human operators from being hurt by the robots or collaborative robots (cobots), a traditional method is to isolate the robots from the human workers by means of fences and sensors. However, the deployment of deep learning models is unknown and shown to be non-trivial in downstream tasks such as image classification and object detection. The present study aimed to exploit the effectiveness of object detection models, particularly EfficientDet models via a transfer learning approach—fine-tuning. A total of 1463 images were obtained from the surveillance cameras from TT Vision Holdings Berhad and split into training, validation, and test by a ratio of 70:20:10. The training images were further augmented using horizontal flip and scale jittering techniques to increase the total training images up to 3072 images. As an outcome, the result revealed that the EfficientDet-D2 fine-tuned model achieved a test AP of 81.70% with an inference speed of 97.06 ms on Tesla T4 while the EfficientDet-D0 fine-tuned model attained a test AP of 69.30% with an inference speed of 30.24 ms on Tesla T4. In comparison between the EfficientDet-D0 fine-tuned model and EfficientDet-D2 fine-tuned model, the performance improved in terms of AP with the inference speed as the trade-off. The research has shown that it is feasible to detect the presence of human workers and can possibly serve as the visual perception of the robot with regards to human presence detection. Last but not least, the present work proved the applicability of transfer learning methods in human presence detection, specifically fine-tuned object detection models. |
| first_indexed | 2024-03-06T13:04:53Z |
| format | Book Chapter |
| id | UMPir37059 |
| institution | Universiti Malaysia Pahang |
| language | English |
| last_indexed | 2024-03-06T13:04:53Z |
| publishDate | 2022 |
| publisher | Springer, Singapore |
| record_format | dspace |
| spelling | UMPir370592023-03-09T04:41:53Z http://umpir.ump.edu.my/id/eprint/37059/ Vision-based Human Presence Detection by Means of Transfer Learning Approach Tang, Jin Cheng Ahmad Fakhri, Ab. Nasir Anwar, P. P. Abdul Majeed Mohd Azraai, Mohd Razman Ismail, Mohd Khairuddin Thai, Li Lim QA76 Computer software T Technology (General) TK Electrical engineering. Electronics Nuclear engineering Human–robot interaction (HRI) and human–robot collaboration (HRC) has become more important as the industries are stepping towards the phase of digitalization and Industry 4.0. Indeed, the emphasis is often placed on the safety of the physical well-being of the human workers. To safeguard the human operators from being hurt by the robots or collaborative robots (cobots), a traditional method is to isolate the robots from the human workers by means of fences and sensors. However, the deployment of deep learning models is unknown and shown to be non-trivial in downstream tasks such as image classification and object detection. The present study aimed to exploit the effectiveness of object detection models, particularly EfficientDet models via a transfer learning approach—fine-tuning. A total of 1463 images were obtained from the surveillance cameras from TT Vision Holdings Berhad and split into training, validation, and test by a ratio of 70:20:10. The training images were further augmented using horizontal flip and scale jittering techniques to increase the total training images up to 3072 images. As an outcome, the result revealed that the EfficientDet-D2 fine-tuned model achieved a test AP of 81.70% with an inference speed of 97.06 ms on Tesla T4 while the EfficientDet-D0 fine-tuned model attained a test AP of 69.30% with an inference speed of 30.24 ms on Tesla T4. In comparison between the EfficientDet-D0 fine-tuned model and EfficientDet-D2 fine-tuned model, the performance improved in terms of AP with the inference speed as the trade-off. The research has shown that it is feasible to detect the presence of human workers and can possibly serve as the visual perception of the robot with regards to human presence detection. Last but not least, the present work proved the applicability of transfer learning methods in human presence detection, specifically fine-tuned object detection models. Springer, Singapore 2022-05-15 Book Chapter PeerReviewed pdf en http://umpir.ump.edu.my/id/eprint/37059/1/Vision-Based%20Human%20Presence%20Detection%20by%20Means%20of%20Transfer%20Learning%20Approach.pdf Tang, Jin Cheng and Ahmad Fakhri, Ab. Nasir and Anwar, P. P. Abdul Majeed and Mohd Azraai, Mohd Razman and Ismail, Mohd Khairuddin and Thai, Li Lim (2022) Vision-based Human Presence Detection by Means of Transfer Learning Approach. In: Enabling Industry 4.0 through Advances in Mechatronics. Lecture Notes in Electrical Engineering, 900 . Springer, Singapore, Singapore, pp. 571-580. ISBN 978-981-19-2094-3 (Printed); 978-981-19-2095-0 (Online) https://doi.org/10.1007/978-981-19-2095-0_49 http://:10.1007/978-981-19-2095-0_49 |
| spellingShingle | QA76 Computer software T Technology (General) TK Electrical engineering. Electronics Nuclear engineering Tang, Jin Cheng Ahmad Fakhri, Ab. Nasir Anwar, P. P. Abdul Majeed Mohd Azraai, Mohd Razman Ismail, Mohd Khairuddin Thai, Li Lim Vision-based Human Presence Detection by Means of Transfer Learning Approach |
| title | Vision-based Human Presence Detection by Means of Transfer Learning Approach |
| title_full | Vision-based Human Presence Detection by Means of Transfer Learning Approach |
| title_fullStr | Vision-based Human Presence Detection by Means of Transfer Learning Approach |
| title_full_unstemmed | Vision-based Human Presence Detection by Means of Transfer Learning Approach |
| title_short | Vision-based Human Presence Detection by Means of Transfer Learning Approach |
| title_sort | vision based human presence detection by means of transfer learning approach |
| topic | QA76 Computer software T Technology (General) TK Electrical engineering. Electronics Nuclear engineering |
| url | http://umpir.ump.edu.my/id/eprint/37059/1/Vision-Based%20Human%20Presence%20Detection%20by%20Means%20of%20Transfer%20Learning%20Approach.pdf |
| work_keys_str_mv | AT tangjincheng visionbasedhumanpresencedetectionbymeansoftransferlearningapproach AT ahmadfakhriabnasir visionbasedhumanpresencedetectionbymeansoftransferlearningapproach AT anwarppabdulmajeed visionbasedhumanpresencedetectionbymeansoftransferlearningapproach AT mohdazraaimohdrazman visionbasedhumanpresencedetectionbymeansoftransferlearningapproach AT ismailmohdkhairuddin visionbasedhumanpresencedetectionbymeansoftransferlearningapproach AT thaililim visionbasedhumanpresencedetectionbymeansoftransferlearningapproach |