Tea Bud Detection and 3D Pose Estimation in the Field with a Depth Camera Based on Improved YOLOv5 and the Optimal Pose-Vertices Search Method
The precise detection and positioning of tea buds are among the major issues in tea picking automation. In this study, a novel algorithm for detecting tea buds and estimating their poses in a field environment was proposed by using a depth camera. This algorithm introduces some improvements to the Y...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-07-01
|
| Series: | Agriculture |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2077-0472/13/7/1405 |
| _version_ | 1827734399532138496 |
|---|---|
| author | Zhiwei Chen Jianneng Chen Yang Li Zhiyong Gui Taojie Yu |
| author_facet | Zhiwei Chen Jianneng Chen Yang Li Zhiyong Gui Taojie Yu |
| author_sort | Zhiwei Chen |
| collection | DOAJ |
| description | The precise detection and positioning of tea buds are among the major issues in tea picking automation. In this study, a novel algorithm for detecting tea buds and estimating their poses in a field environment was proposed by using a depth camera. This algorithm introduces some improvements to the YOLOv5l architecture. A Coordinate Attention Mechanism (CAM) was inserted into the neck part to accurately position the elements of interest, a BiFPN was used to enhance the small object detection ability, and a GhostConv module replaced the original Conv module in the backbone to reduce the model size and speed up model inference. After testing, the proposed detection model achieved an mAP of 85.2%, a speed of 87.71 FPS, a parameter number of 29.25 M, and a FLOPs value of 59.8 G, which are all better than those achieved with the original model. Next, an optimal pose-vertices search method (OPVSM) was developed to estimate the pose of tea by constructing a graph model to fit the pointcloud. This method could accurately estimate the poses of tea buds, with an overall accuracy of 90%, and it was more flexible and adaptive to the variations in tea buds in terms of size, color, and shape features. Additionally, the experiments demonstrated that the OPVSM could correctly establish the pose of tea buds through pointcloud downsampling by using voxel filtering with a 2 mm × 2 mm × 1 mm grid, and this process could effectively reduce the size of the pointcloud to smaller than 800 to ensure that the algorithm could be run within 0.2 s. The results demonstrate the effectiveness of the proposed algorithm for tea bud detection and pose estimation in a field setting. Furthermore, the proposed algorithm has the potential to be used in tea picking robots and also can be extended to other crops and objects, making it a valuable tool for precision agriculture and robotic applications. |
| first_indexed | 2024-03-11T01:24:42Z |
| format | Article |
| id | doaj.art-aab3326c48104c8da59e61e19b0d56d3 |
| institution | Directory Open Access Journal |
| issn | 2077-0472 |
| language | English |
| last_indexed | 2024-03-11T01:24:42Z |
| publishDate | 2023-07-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Agriculture |
| spelling | doaj.art-aab3326c48104c8da59e61e19b0d56d32023-11-18T17:53:24ZengMDPI AGAgriculture2077-04722023-07-01137140510.3390/agriculture13071405Tea Bud Detection and 3D Pose Estimation in the Field with a Depth Camera Based on Improved YOLOv5 and the Optimal Pose-Vertices Search MethodZhiwei Chen0Jianneng Chen1Yang Li2Zhiyong Gui3Taojie Yu4School of Mechanical Engineering, Zhejiang Sci-Tech University, Hangzhou 310018, ChinaSchool of Mechanical Engineering, Zhejiang Sci-Tech University, Hangzhou 310018, ChinaTea Research Institute, Chinese Academy of Agricultural Sciences, Hangzhou 310008, ChinaSchool of Mechanical Engineering, Zhejiang Sci-Tech University, Hangzhou 310018, ChinaSchool of Mechanical Engineering, Zhejiang Sci-Tech University, Hangzhou 310018, ChinaThe precise detection and positioning of tea buds are among the major issues in tea picking automation. In this study, a novel algorithm for detecting tea buds and estimating their poses in a field environment was proposed by using a depth camera. This algorithm introduces some improvements to the YOLOv5l architecture. A Coordinate Attention Mechanism (CAM) was inserted into the neck part to accurately position the elements of interest, a BiFPN was used to enhance the small object detection ability, and a GhostConv module replaced the original Conv module in the backbone to reduce the model size and speed up model inference. After testing, the proposed detection model achieved an mAP of 85.2%, a speed of 87.71 FPS, a parameter number of 29.25 M, and a FLOPs value of 59.8 G, which are all better than those achieved with the original model. Next, an optimal pose-vertices search method (OPVSM) was developed to estimate the pose of tea by constructing a graph model to fit the pointcloud. This method could accurately estimate the poses of tea buds, with an overall accuracy of 90%, and it was more flexible and adaptive to the variations in tea buds in terms of size, color, and shape features. Additionally, the experiments demonstrated that the OPVSM could correctly establish the pose of tea buds through pointcloud downsampling by using voxel filtering with a 2 mm × 2 mm × 1 mm grid, and this process could effectively reduce the size of the pointcloud to smaller than 800 to ensure that the algorithm could be run within 0.2 s. The results demonstrate the effectiveness of the proposed algorithm for tea bud detection and pose estimation in a field setting. Furthermore, the proposed algorithm has the potential to be used in tea picking robots and also can be extended to other crops and objects, making it a valuable tool for precision agriculture and robotic applications.https://www.mdpi.com/2077-0472/13/7/1405tea bud detectionYOLOv5depth camerapose estimationCAMOPVSM |
| spellingShingle | Zhiwei Chen Jianneng Chen Yang Li Zhiyong Gui Taojie Yu Tea Bud Detection and 3D Pose Estimation in the Field with a Depth Camera Based on Improved YOLOv5 and the Optimal Pose-Vertices Search Method Agriculture tea bud detection YOLOv5 depth camera pose estimation CAM OPVSM |
| title | Tea Bud Detection and 3D Pose Estimation in the Field with a Depth Camera Based on Improved YOLOv5 and the Optimal Pose-Vertices Search Method |
| title_full | Tea Bud Detection and 3D Pose Estimation in the Field with a Depth Camera Based on Improved YOLOv5 and the Optimal Pose-Vertices Search Method |
| title_fullStr | Tea Bud Detection and 3D Pose Estimation in the Field with a Depth Camera Based on Improved YOLOv5 and the Optimal Pose-Vertices Search Method |
| title_full_unstemmed | Tea Bud Detection and 3D Pose Estimation in the Field with a Depth Camera Based on Improved YOLOv5 and the Optimal Pose-Vertices Search Method |
| title_short | Tea Bud Detection and 3D Pose Estimation in the Field with a Depth Camera Based on Improved YOLOv5 and the Optimal Pose-Vertices Search Method |
| title_sort | tea bud detection and 3d pose estimation in the field with a depth camera based on improved yolov5 and the optimal pose vertices search method |
| topic | tea bud detection YOLOv5 depth camera pose estimation CAM OPVSM |
| url | https://www.mdpi.com/2077-0472/13/7/1405 |
| work_keys_str_mv | AT zhiweichen teabuddetectionand3dposeestimationinthefieldwithadepthcamerabasedonimprovedyolov5andtheoptimalposeverticessearchmethod AT jiannengchen teabuddetectionand3dposeestimationinthefieldwithadepthcamerabasedonimprovedyolov5andtheoptimalposeverticessearchmethod AT yangli teabuddetectionand3dposeestimationinthefieldwithadepthcamerabasedonimprovedyolov5andtheoptimalposeverticessearchmethod AT zhiyonggui teabuddetectionand3dposeestimationinthefieldwithadepthcamerabasedonimprovedyolov5andtheoptimalposeverticessearchmethod AT taojieyu teabuddetectionand3dposeestimationinthefieldwithadepthcamerabasedonimprovedyolov5andtheoptimalposeverticessearchmethod |