Vision-based human presence detection pipeline by means of transfer learning approach
Over the last century, industrial robots have gained an immense amount of popularity in replacing the human workers due to their highly repetitive nature. It was a twist to the industries when the concept of cooperative robots, known as cobots, has been innovated. Sharing space between the cobots an...
| Main Author: | |
|---|---|
| Format: | Thesis |
| Language: | English |
| Published: |
2023
|
| Subjects: | |
| Online Access: | http://umpir.ump.edu.my/id/eprint/39286/1/ir.Vision-based%20human%20presence%20detection%20pipeline%20by%20means%20of%20transfer%20learning%20approach.pdf |
| _version_ | 1796996032570064896 |
|---|---|
| author | Tang, Jin Cheng |
| author_facet | Tang, Jin Cheng |
| author_sort | Tang, Jin Cheng |
| collection | UMP |
| description | Over the last century, industrial robots have gained an immense amount of popularity in replacing the human workers due to their highly repetitive nature. It was a twist to the industries when the concept of cooperative robots, known as cobots, has been innovated. Sharing space between the cobots and human workers has considered as the most effective way of utilizing the cobots. Keeping in mind that the safety of the human workers is always the top priority of the cobot applications in the industries, many time and efforts have been invested to improve the safeness of the cobots deployments. Yet, the utilization of deep learning technologies is rarely found in accordance with human detection in the field of research, especially the transfer learning approach, providing that the visual perception has shown to be a unique sense that still cannot be replaced by other. Hence, this thesis aimed to leverage the transfer learning approach to fine-tune the deep learning-based object detection models in the human detection task. In relation to this main goal, the objectives of the study were as follows: establish an image dataset for cobot environment from the surveillance cameras in TT Vision Holdings Berhad, formulate deep learning-based object detection models by using the transfer learning approach, and evaluate the performance of various transfer learning models in detecting the presence of human workers with relevant evaluation metrics. Image dataset has acquired from the surveillance system of TT Vision Holdings Berhad and annotated accordingly. The variations of the dataset have been considered thoroughly to ensure the models can be well-trained on the distinct features of the human workers. TensorFlow Object Detection API was used in the study to perform the fine-tuning of the one-stage object detectors. Among all the transfer learning strategies, fine-tuning has chosen since it suits the study well after the interpretation on the size-similarity matrix. A total of four EfficientDet models, two SSD models, three RetinaNet models, and four CenterNet models were deployed in the present work. As a result, SSD-MobileNetV2-FPN model has achieved 81.1% AP with 32.82 FPS, which is proposed as the best well-balanced fine-tuned model between accuracy and speed. In other case where the consideration is taken solely on either accuracy or inference speed, SSD_MobileNetV1-FPN model has attained 87.2% AP with 28.28 FPS and CenterNet-ResNet50-V1-FPN has achieved 78.0% AP with 46.52 FPS, which is proposed to be the model with best accuracy and inference speed, respectively. As a whole, it could be deduced that the transfer learning models can handle the human detection task well via the fine-tuning on the COCO-pretrained weights. |
| first_indexed | 2024-03-06T13:10:59Z |
| format | Thesis |
| id | UMPir39286 |
| institution | Universiti Malaysia Pahang |
| language | English |
| last_indexed | 2024-03-06T13:10:59Z |
| publishDate | 2023 |
| record_format | dspace |
| spelling | UMPir392862023-11-14T02:42:04Z http://umpir.ump.edu.my/id/eprint/39286/ Vision-based human presence detection pipeline by means of transfer learning approach Tang, Jin Cheng TA Engineering (General). Civil engineering (General) TS Manufactures Over the last century, industrial robots have gained an immense amount of popularity in replacing the human workers due to their highly repetitive nature. It was a twist to the industries when the concept of cooperative robots, known as cobots, has been innovated. Sharing space between the cobots and human workers has considered as the most effective way of utilizing the cobots. Keeping in mind that the safety of the human workers is always the top priority of the cobot applications in the industries, many time and efforts have been invested to improve the safeness of the cobots deployments. Yet, the utilization of deep learning technologies is rarely found in accordance with human detection in the field of research, especially the transfer learning approach, providing that the visual perception has shown to be a unique sense that still cannot be replaced by other. Hence, this thesis aimed to leverage the transfer learning approach to fine-tune the deep learning-based object detection models in the human detection task. In relation to this main goal, the objectives of the study were as follows: establish an image dataset for cobot environment from the surveillance cameras in TT Vision Holdings Berhad, formulate deep learning-based object detection models by using the transfer learning approach, and evaluate the performance of various transfer learning models in detecting the presence of human workers with relevant evaluation metrics. Image dataset has acquired from the surveillance system of TT Vision Holdings Berhad and annotated accordingly. The variations of the dataset have been considered thoroughly to ensure the models can be well-trained on the distinct features of the human workers. TensorFlow Object Detection API was used in the study to perform the fine-tuning of the one-stage object detectors. Among all the transfer learning strategies, fine-tuning has chosen since it suits the study well after the interpretation on the size-similarity matrix. A total of four EfficientDet models, two SSD models, three RetinaNet models, and four CenterNet models were deployed in the present work. As a result, SSD-MobileNetV2-FPN model has achieved 81.1% AP with 32.82 FPS, which is proposed as the best well-balanced fine-tuned model between accuracy and speed. In other case where the consideration is taken solely on either accuracy or inference speed, SSD_MobileNetV1-FPN model has attained 87.2% AP with 28.28 FPS and CenterNet-ResNet50-V1-FPN has achieved 78.0% AP with 46.52 FPS, which is proposed to be the model with best accuracy and inference speed, respectively. As a whole, it could be deduced that the transfer learning models can handle the human detection task well via the fine-tuning on the COCO-pretrained weights. 2023-06 Thesis NonPeerReviewed pdf en http://umpir.ump.edu.my/id/eprint/39286/1/ir.Vision-based%20human%20presence%20detection%20pipeline%20by%20means%20of%20transfer%20learning%20approach.pdf Tang, Jin Cheng (2023) Vision-based human presence detection pipeline by means of transfer learning approach. Masters thesis, Universiti Malaysia Pahang (Contributors, Thesis advisor: Ahmad Fakhri, Ab. Nasir). |
| spellingShingle | TA Engineering (General). Civil engineering (General) TS Manufactures Tang, Jin Cheng Vision-based human presence detection pipeline by means of transfer learning approach |
| title | Vision-based human presence detection pipeline by means of transfer learning approach |
| title_full | Vision-based human presence detection pipeline by means of transfer learning approach |
| title_fullStr | Vision-based human presence detection pipeline by means of transfer learning approach |
| title_full_unstemmed | Vision-based human presence detection pipeline by means of transfer learning approach |
| title_short | Vision-based human presence detection pipeline by means of transfer learning approach |
| title_sort | vision based human presence detection pipeline by means of transfer learning approach |
| topic | TA Engineering (General). Civil engineering (General) TS Manufactures |
| url | http://umpir.ump.edu.my/id/eprint/39286/1/ir.Vision-based%20human%20presence%20detection%20pipeline%20by%20means%20of%20transfer%20learning%20approach.pdf |
| work_keys_str_mv | AT tangjincheng visionbasedhumanpresencedetectionpipelinebymeansoftransferlearningapproach |