Guava Detection and Pose Estimation Using a Low-Cost RGB-D Sensor in the Field
Fruit detection in real outdoor conditions is necessary for automatic guava harvesting, and the branch-dependent pose of fruits is also crucial to guide a robot to approach and detach the target fruit without colliding with its mother branch. To conduct automatic, collision-free picking, this study...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2019-01-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/19/2/428 |
| _version_ | 1811187579120254976 |
|---|---|
| author | Guichao Lin Yunchao Tang Xiangjun Zou Juntao Xiong Jinhui Li |
| author_facet | Guichao Lin Yunchao Tang Xiangjun Zou Juntao Xiong Jinhui Li |
| author_sort | Guichao Lin |
| collection | DOAJ |
| description | Fruit detection in real outdoor conditions is necessary for automatic guava harvesting, and the branch-dependent pose of fruits is also crucial to guide a robot to approach and detach the target fruit without colliding with its mother branch. To conduct automatic, collision-free picking, this study investigates a fruit detection and pose estimation method by using a low-cost red⁻green⁻blue⁻depth (RGB-D) sensor. A state-of-the-art fully convolutional network is first deployed to segment the RGB image to output a fruit and branch binary map. Based on the fruit binary map and RGB-D depth image, Euclidean clustering is then applied to group the point cloud into a set of individual fruits. Next, a multiple three-dimensional (3D) line-segments detection method is developed to reconstruct the segmented branches. Finally, the 3D pose of the fruit is estimated using its center position and nearest branch information. A dataset was acquired in an outdoor orchard to evaluate the performance of the proposed method. Quantitative experiments showed that the precision and recall of guava fruit detection were 0.983 and 0.948, respectively; the 3D pose error was 23.43° ± 14.18°; and the execution time per fruit was 0.565 s. The results demonstrate that the developed method can be applied to a guava-harvesting robot. |
| first_indexed | 2024-04-11T14:04:48Z |
| format | Article |
| id | doaj.art-607ec852a85649959b142e23a1ea7fcd |
| institution | Directory Open Access Journal |
| issn | 1424-8220 |
| language | English |
| last_indexed | 2024-04-11T14:04:48Z |
| publishDate | 2019-01-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj.art-607ec852a85649959b142e23a1ea7fcd2022-12-22T04:19:55ZengMDPI AGSensors1424-82202019-01-0119242810.3390/s19020428s19020428Guava Detection and Pose Estimation Using a Low-Cost RGB-D Sensor in the FieldGuichao Lin0Yunchao Tang1Xiangjun Zou2Juntao Xiong3Jinhui Li4Key Laboratory of Key Technology on Agricultural Machine and Equipment, Ministry of Education, South China Agricultural University, Guangzhou 510642, ChinaSchool of Urban and Rural Construction, Zhongkai University of Agriculture and Engineering, Guangzhou 510006, ChinaKey Laboratory of Key Technology on Agricultural Machine and Equipment, Ministry of Education, South China Agricultural University, Guangzhou 510642, ChinaKey Laboratory of Key Technology on Agricultural Machine and Equipment, Ministry of Education, South China Agricultural University, Guangzhou 510642, ChinaKey Laboratory of Key Technology on Agricultural Machine and Equipment, Ministry of Education, South China Agricultural University, Guangzhou 510642, ChinaFruit detection in real outdoor conditions is necessary for automatic guava harvesting, and the branch-dependent pose of fruits is also crucial to guide a robot to approach and detach the target fruit without colliding with its mother branch. To conduct automatic, collision-free picking, this study investigates a fruit detection and pose estimation method by using a low-cost red⁻green⁻blue⁻depth (RGB-D) sensor. A state-of-the-art fully convolutional network is first deployed to segment the RGB image to output a fruit and branch binary map. Based on the fruit binary map and RGB-D depth image, Euclidean clustering is then applied to group the point cloud into a set of individual fruits. Next, a multiple three-dimensional (3D) line-segments detection method is developed to reconstruct the segmented branches. Finally, the 3D pose of the fruit is estimated using its center position and nearest branch information. A dataset was acquired in an outdoor orchard to evaluate the performance of the proposed method. Quantitative experiments showed that the precision and recall of guava fruit detection were 0.983 and 0.948, respectively; the 3D pose error was 23.43° ± 14.18°; and the execution time per fruit was 0.565 s. The results demonstrate that the developed method can be applied to a guava-harvesting robot.https://www.mdpi.com/1424-8220/19/2/428guava detectionpose estimationfully convolutional networkbranch reconstructionRGB-D sensor |
| spellingShingle | Guichao Lin Yunchao Tang Xiangjun Zou Juntao Xiong Jinhui Li Guava Detection and Pose Estimation Using a Low-Cost RGB-D Sensor in the Field Sensors guava detection pose estimation fully convolutional network branch reconstruction RGB-D sensor |
| title | Guava Detection and Pose Estimation Using a Low-Cost RGB-D Sensor in the Field |
| title_full | Guava Detection and Pose Estimation Using a Low-Cost RGB-D Sensor in the Field |
| title_fullStr | Guava Detection and Pose Estimation Using a Low-Cost RGB-D Sensor in the Field |
| title_full_unstemmed | Guava Detection and Pose Estimation Using a Low-Cost RGB-D Sensor in the Field |
| title_short | Guava Detection and Pose Estimation Using a Low-Cost RGB-D Sensor in the Field |
| title_sort | guava detection and pose estimation using a low cost rgb d sensor in the field |
| topic | guava detection pose estimation fully convolutional network branch reconstruction RGB-D sensor |
| url | https://www.mdpi.com/1424-8220/19/2/428 |
| work_keys_str_mv | AT guichaolin guavadetectionandposeestimationusingalowcostrgbdsensorinthefield AT yunchaotang guavadetectionandposeestimationusingalowcostrgbdsensorinthefield AT xiangjunzou guavadetectionandposeestimationusingalowcostrgbdsensorinthefield AT juntaoxiong guavadetectionandposeestimationusingalowcostrgbdsensorinthefield AT jinhuili guavadetectionandposeestimationusingalowcostrgbdsensorinthefield |