An Object Detection and Localization Method Based on Improved YOLOv5 for the Teleoperated Robot
In the traditional teleoperation system, the operator locates the object using the real-time scene information sent back from the robot terminal; however, the localization accuracy is poor and the execution efficiency is low. To address the issues, we propose an object detection and localization met...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2022-11-01
|
| Series: | Applied Sciences |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2076-3417/12/22/11441 |
| _version_ | 1797466005863137280 |
|---|---|
| author | Zhangyi Chen Xiaoling Li Long Wang Yueyang Shi Zhipeng Sun Wei Sun |
| author_facet | Zhangyi Chen Xiaoling Li Long Wang Yueyang Shi Zhipeng Sun Wei Sun |
| author_sort | Zhangyi Chen |
| collection | DOAJ |
| description | In the traditional teleoperation system, the operator locates the object using the real-time scene information sent back from the robot terminal; however, the localization accuracy is poor and the execution efficiency is low. To address the issues, we propose an object detection and localization method for the teleoperated robot. First, we improved the classic YOLOv5 network model to produce superior object detection performance and named the improved model YOLOv5_Tel. On the basis of the classic YOLOv5 network model, the feature pyramid network was changed to a bidirectional feature pyramid network (BiFPN) network module to achieve the weighted feature fusion mechanism. The coordinate attention (CA) module was added to make the model pay more attention to the features of interest. Furthermore, we pruned the model from the depth and width to make it more lightweight and changed the bounding box regression loss function GIOU to SIOU to speed up model convergence. Then, the YOLOv5_Tel model and ZED2 depth camera were used to achieve object localization based on the binocular stereo vision ranging principle. Finally, we established an object detection platform for the teleoperated robot and created a small dataset to validate the proposed method. The experiment shows that compared with the classic YOLOv5 series network model, the YOLOv5_Tel is higher in accuracy, lighter in weight, and faster in detection speed. The mean average precision (mAP) value of the YOLOv5_Tel increased by 0.8%, 0.9%, and 1.0%, respectively. The model size decreased by 11.1%, 70.0%, and 86.4%, respectively. The inference time decreased by 9.1%, 42.9%, and 58.3%, respectively. The proposed object localization method has a high localization accuracy with an average relative error of only 1.12%. |
| first_indexed | 2024-03-09T18:30:39Z |
| format | Article |
| id | doaj.art-c472497ffc194a139c4b0c125eb2d81d |
| institution | Directory Open Access Journal |
| issn | 2076-3417 |
| language | English |
| last_indexed | 2024-03-09T18:30:39Z |
| publishDate | 2022-11-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Applied Sciences |
| spelling | doaj.art-c472497ffc194a139c4b0c125eb2d81d2023-11-24T07:35:19ZengMDPI AGApplied Sciences2076-34172022-11-0112221144110.3390/app122211441An Object Detection and Localization Method Based on Improved YOLOv5 for the Teleoperated RobotZhangyi Chen0Xiaoling Li1Long Wang2Yueyang Shi3Zhipeng Sun4Wei Sun5School of Mechanical Engineering, Xi’an Jiaotong University, Xi’an 710000, ChinaSchool of Mechanical Engineering, Xi’an Jiaotong University, Xi’an 710000, ChinaSchool of Mechanical Engineering, Xi’an Jiaotong University, Xi’an 710000, ChinaSchool of Mechanical Engineering, Xi’an Jiaotong University, Xi’an 710000, ChinaSchool of Mechanical Engineering, Xi’an Jiaotong University, Xi’an 710000, ChinaSchool of Mechanical Engineering, Xi’an Jiaotong University, Xi’an 710000, ChinaIn the traditional teleoperation system, the operator locates the object using the real-time scene information sent back from the robot terminal; however, the localization accuracy is poor and the execution efficiency is low. To address the issues, we propose an object detection and localization method for the teleoperated robot. First, we improved the classic YOLOv5 network model to produce superior object detection performance and named the improved model YOLOv5_Tel. On the basis of the classic YOLOv5 network model, the feature pyramid network was changed to a bidirectional feature pyramid network (BiFPN) network module to achieve the weighted feature fusion mechanism. The coordinate attention (CA) module was added to make the model pay more attention to the features of interest. Furthermore, we pruned the model from the depth and width to make it more lightweight and changed the bounding box regression loss function GIOU to SIOU to speed up model convergence. Then, the YOLOv5_Tel model and ZED2 depth camera were used to achieve object localization based on the binocular stereo vision ranging principle. Finally, we established an object detection platform for the teleoperated robot and created a small dataset to validate the proposed method. The experiment shows that compared with the classic YOLOv5 series network model, the YOLOv5_Tel is higher in accuracy, lighter in weight, and faster in detection speed. The mean average precision (mAP) value of the YOLOv5_Tel increased by 0.8%, 0.9%, and 1.0%, respectively. The model size decreased by 11.1%, 70.0%, and 86.4%, respectively. The inference time decreased by 9.1%, 42.9%, and 58.3%, respectively. The proposed object localization method has a high localization accuracy with an average relative error of only 1.12%.https://www.mdpi.com/2076-3417/12/22/11441teleoperated robotobject detectionobject localizationimproved YOLOv5 networkdistance estimation |
| spellingShingle | Zhangyi Chen Xiaoling Li Long Wang Yueyang Shi Zhipeng Sun Wei Sun An Object Detection and Localization Method Based on Improved YOLOv5 for the Teleoperated Robot Applied Sciences teleoperated robot object detection object localization improved YOLOv5 network distance estimation |
| title | An Object Detection and Localization Method Based on Improved YOLOv5 for the Teleoperated Robot |
| title_full | An Object Detection and Localization Method Based on Improved YOLOv5 for the Teleoperated Robot |
| title_fullStr | An Object Detection and Localization Method Based on Improved YOLOv5 for the Teleoperated Robot |
| title_full_unstemmed | An Object Detection and Localization Method Based on Improved YOLOv5 for the Teleoperated Robot |
| title_short | An Object Detection and Localization Method Based on Improved YOLOv5 for the Teleoperated Robot |
| title_sort | object detection and localization method based on improved yolov5 for the teleoperated robot |
| topic | teleoperated robot object detection object localization improved YOLOv5 network distance estimation |
| url | https://www.mdpi.com/2076-3417/12/22/11441 |
| work_keys_str_mv | AT zhangyichen anobjectdetectionandlocalizationmethodbasedonimprovedyolov5fortheteleoperatedrobot AT xiaolingli anobjectdetectionandlocalizationmethodbasedonimprovedyolov5fortheteleoperatedrobot AT longwang anobjectdetectionandlocalizationmethodbasedonimprovedyolov5fortheteleoperatedrobot AT yueyangshi anobjectdetectionandlocalizationmethodbasedonimprovedyolov5fortheteleoperatedrobot AT zhipengsun anobjectdetectionandlocalizationmethodbasedonimprovedyolov5fortheteleoperatedrobot AT weisun anobjectdetectionandlocalizationmethodbasedonimprovedyolov5fortheteleoperatedrobot AT zhangyichen objectdetectionandlocalizationmethodbasedonimprovedyolov5fortheteleoperatedrobot AT xiaolingli objectdetectionandlocalizationmethodbasedonimprovedyolov5fortheteleoperatedrobot AT longwang objectdetectionandlocalizationmethodbasedonimprovedyolov5fortheteleoperatedrobot AT yueyangshi objectdetectionandlocalizationmethodbasedonimprovedyolov5fortheteleoperatedrobot AT zhipengsun objectdetectionandlocalizationmethodbasedonimprovedyolov5fortheteleoperatedrobot AT weisun objectdetectionandlocalizationmethodbasedonimprovedyolov5fortheteleoperatedrobot |