Rapid Recognition and Picking Points Automatic Positioning Method for Table Grape in Natural Environment
ObjectiveRapid recognition and automatic positioning of table grapes in the natural environment is the prerequisite for the automatic picking of table grapes by the picking robot.MethodsAn rapid recognition and automatic picking points positioning method based on improved K-means clustering algorith...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Editorial Office of Smart Agriculture
2023-06-01
|
| Series: | 智慧农业 |
| Subjects: | |
| Online Access: | http://www.smartag.net.cn/CN/10.12133/j.smartag.SA202304001 |
| _version_ | 1827877522508873728 |
|---|---|
| author | ZHU Yanjun DU Wensheng WANG Chunying LIU Ping LI Xiang |
| author_facet | ZHU Yanjun DU Wensheng WANG Chunying LIU Ping LI Xiang |
| author_sort | ZHU Yanjun |
| collection | DOAJ |
| description | ObjectiveRapid recognition and automatic positioning of table grapes in the natural environment is the prerequisite for the automatic picking of table grapes by the picking robot.MethodsAn rapid recognition and automatic picking points positioning method based on improved K-means clustering algorithm and contour analysis was proposed. First, euclidean distance was replaced by a weighted gray threshold as the judgment basis of K-means similarity. Then the images of table grapes were rasterized according to the K value, and the initial clustering center was obtained. Next, the average gray value of each cluster and the percentage of pixel points of each cluster in the total pixel points were calculated. And the weighted gray threshold was obtained by the average gray value and percentage of adjacent clusters. Then, the clustering was considered as have ended until the weighted gray threshold remained unchanged. Therefore, the cluster image of table grape was obtained. The improved clustering algorithm not only saved the clustering time, but also ensured that the K value could change adaptively. Moreover, the adaptive Otsu algorithm was used to extract grape cluster information, so that the initial binary image of the table grape was obtained. In order to reduce the interference of redundant noise on recognition accuracy, the morphological algorithms (open operation, close operation, images filling and the maximum connected domain) were used to remove noise, so the accurate binary image of table grapes was obtained. And then, the contours of table grapes were obtained by the Sobel operator. Furthermore, table grape clusters grew perpendicular to the ground due to gravity in the natural environment. Therefore, the extreme point and center of gravity point of the grape cluster were obtained based on contour analysis. In addition, the linear bundle where the extreme point and the center of gravity point located was taken as the carrier, and the similarity of pixel points on both sides of the linear bundle were taken as the judgment basis. The line corresponding to the lowest similarity value was taken as the grape stem, so the stem axis of the grape was located. Moreover, according to the agronomic picking requirements of table grapes, and combined with contour analysis, the region of interest (ROI) in picking points could be obtained. Among them, the intersection of the grapes stem and the contour was regarded as the middle point of the bottom edge of the ROI. And the 0.8 times distance between the left and right extreme points was regarded as the length of the ROI, the 0.25 times distance between the gravity point and the intersection of the grape stem and the contour was regarded as the height of the ROI. After that, the central point of the ROI was captured. Then, the nearest point between the center point of the ROI and the grape stem was determined, and this point on the grape stem was taken as the picking point of the table grapes. Finally, 917 grape images (including Summer Black, Moldova, and Youyong) taken by the rear camera of MI8 mobile phone at Jinniu Mountain Base of Shandong Fruit and Vegetable Research Institute were verified experimentally.Results and Discussions]The results showed that the success rate was 90.51% when the error between the table grape picking points and the optimal points were less than 12 pixels, and the average positioning time was 0.87 s. The method realized the fast and accurate localization of table grape picking points. On top of that, according to the two cultivation modes (hedgerow planting and trellis planting) of table grapes, a simulation test platform based on the Dense mechanical arm and the single-chip computer was set up in the study. 50 simulation tests were carried out for the four conditions respectively, among which the success rate of localization for purple grape picking point of hedgerow planting was 86.00%, and the average localization time was 0.89 s; the success rate of localization for purple grape identification and localization of trellis planting was 92.00%, and the average localization time was 0.67 s; the success rate of localization for green grape picking point of hedgerow planting was 78.00%, and the average localization time was 0.72 s; and the success rate of localization for green grape identification and localization of trellis planting was 80.00%, and the average localization time was 0.71 s.ConclusionsThe experimental results showed that the method proposed in the study can meet the requirements of table grape picking, and can provide technical supports for the development of grape picking robot. |
| first_indexed | 2024-03-12T17:37:42Z |
| format | Article |
| id | doaj.art-70f6b55408c642c58115940377e3f1dd |
| institution | Directory Open Access Journal |
| issn | 2096-8094 |
| language | English |
| last_indexed | 2024-03-12T17:37:42Z |
| publishDate | 2023-06-01 |
| publisher | Editorial Office of Smart Agriculture |
| record_format | Article |
| series | 智慧农业 |
| spelling | doaj.art-70f6b55408c642c58115940377e3f1dd2023-08-04T06:21:50ZengEditorial Office of Smart Agriculture智慧农业2096-80942023-06-0152233410.12133/j.smartag.SA202304001SA202304001Rapid Recognition and Picking Points Automatic Positioning Method for Table Grape in Natural EnvironmentZHU Yanjun0DU Wensheng1WANG Chunying2LIU Ping3LI Xiang4Shandong Agricultural Equipment Intelligent Engineering Laboratory/ Shandong Provincial Key Laboratory of Horticultural/ Machinery and Equipment, College of Mechanical and Electronic Engineering, Shandong Agricultural University, Taian 271018, ChinaShandong Agricultural Equipment Intelligent Engineering Laboratory/ Shandong Provincial Key Laboratory of Horticultural/ Machinery and Equipment, College of Mechanical and Electronic Engineering, Shandong Agricultural University, Taian 271018, ChinaShandong Agricultural Equipment Intelligent Engineering Laboratory/ Shandong Provincial Key Laboratory of Horticultural/ Machinery and Equipment, College of Mechanical and Electronic Engineering, Shandong Agricultural University, Taian 271018, ChinaShandong Agricultural Equipment Intelligent Engineering Laboratory/ Shandong Provincial Key Laboratory of Horticultural/ Machinery and Equipment, College of Mechanical and Electronic Engineering, Shandong Agricultural University, Taian 271018, ChinaNational Key Laboratory of Wheat Improvement, College of Life Sciences, Shandong Agricultural University, Taian 271018, ChinaObjectiveRapid recognition and automatic positioning of table grapes in the natural environment is the prerequisite for the automatic picking of table grapes by the picking robot.MethodsAn rapid recognition and automatic picking points positioning method based on improved K-means clustering algorithm and contour analysis was proposed. First, euclidean distance was replaced by a weighted gray threshold as the judgment basis of K-means similarity. Then the images of table grapes were rasterized according to the K value, and the initial clustering center was obtained. Next, the average gray value of each cluster and the percentage of pixel points of each cluster in the total pixel points were calculated. And the weighted gray threshold was obtained by the average gray value and percentage of adjacent clusters. Then, the clustering was considered as have ended until the weighted gray threshold remained unchanged. Therefore, the cluster image of table grape was obtained. The improved clustering algorithm not only saved the clustering time, but also ensured that the K value could change adaptively. Moreover, the adaptive Otsu algorithm was used to extract grape cluster information, so that the initial binary image of the table grape was obtained. In order to reduce the interference of redundant noise on recognition accuracy, the morphological algorithms (open operation, close operation, images filling and the maximum connected domain) were used to remove noise, so the accurate binary image of table grapes was obtained. And then, the contours of table grapes were obtained by the Sobel operator. Furthermore, table grape clusters grew perpendicular to the ground due to gravity in the natural environment. Therefore, the extreme point and center of gravity point of the grape cluster were obtained based on contour analysis. In addition, the linear bundle where the extreme point and the center of gravity point located was taken as the carrier, and the similarity of pixel points on both sides of the linear bundle were taken as the judgment basis. The line corresponding to the lowest similarity value was taken as the grape stem, so the stem axis of the grape was located. Moreover, according to the agronomic picking requirements of table grapes, and combined with contour analysis, the region of interest (ROI) in picking points could be obtained. Among them, the intersection of the grapes stem and the contour was regarded as the middle point of the bottom edge of the ROI. And the 0.8 times distance between the left and right extreme points was regarded as the length of the ROI, the 0.25 times distance between the gravity point and the intersection of the grape stem and the contour was regarded as the height of the ROI. After that, the central point of the ROI was captured. Then, the nearest point between the center point of the ROI and the grape stem was determined, and this point on the grape stem was taken as the picking point of the table grapes. Finally, 917 grape images (including Summer Black, Moldova, and Youyong) taken by the rear camera of MI8 mobile phone at Jinniu Mountain Base of Shandong Fruit and Vegetable Research Institute were verified experimentally.Results and Discussions]The results showed that the success rate was 90.51% when the error between the table grape picking points and the optimal points were less than 12 pixels, and the average positioning time was 0.87 s. The method realized the fast and accurate localization of table grape picking points. On top of that, according to the two cultivation modes (hedgerow planting and trellis planting) of table grapes, a simulation test platform based on the Dense mechanical arm and the single-chip computer was set up in the study. 50 simulation tests were carried out for the four conditions respectively, among which the success rate of localization for purple grape picking point of hedgerow planting was 86.00%, and the average localization time was 0.89 s; the success rate of localization for purple grape identification and localization of trellis planting was 92.00%, and the average localization time was 0.67 s; the success rate of localization for green grape picking point of hedgerow planting was 78.00%, and the average localization time was 0.72 s; and the success rate of localization for green grape identification and localization of trellis planting was 80.00%, and the average localization time was 0.71 s.ConclusionsThe experimental results showed that the method proposed in the study can meet the requirements of table grape picking, and can provide technical supports for the development of grape picking robot.http://www.smartag.net.cn/CN/10.12133/j.smartag.SA202304001table grapek-meanscontour analysis methodfruit stem axispicking pointpicking robot |
| spellingShingle | ZHU Yanjun DU Wensheng WANG Chunying LIU Ping LI Xiang Rapid Recognition and Picking Points Automatic Positioning Method for Table Grape in Natural Environment 智慧农业 table grape k-means contour analysis method fruit stem axis picking point picking robot |
| title | Rapid Recognition and Picking Points Automatic Positioning Method for Table Grape in Natural Environment |
| title_full | Rapid Recognition and Picking Points Automatic Positioning Method for Table Grape in Natural Environment |
| title_fullStr | Rapid Recognition and Picking Points Automatic Positioning Method for Table Grape in Natural Environment |
| title_full_unstemmed | Rapid Recognition and Picking Points Automatic Positioning Method for Table Grape in Natural Environment |
| title_short | Rapid Recognition and Picking Points Automatic Positioning Method for Table Grape in Natural Environment |
| title_sort | rapid recognition and picking points automatic positioning method for table grape in natural environment |
| topic | table grape k-means contour analysis method fruit stem axis picking point picking robot |
| url | http://www.smartag.net.cn/CN/10.12133/j.smartag.SA202304001 |
| work_keys_str_mv | AT zhuyanjun rapidrecognitionandpickingpointsautomaticpositioningmethodfortablegrapeinnaturalenvironment AT duwensheng rapidrecognitionandpickingpointsautomaticpositioningmethodfortablegrapeinnaturalenvironment AT wangchunying rapidrecognitionandpickingpointsautomaticpositioningmethodfortablegrapeinnaturalenvironment AT liuping rapidrecognitionandpickingpointsautomaticpositioningmethodfortablegrapeinnaturalenvironment AT lixiang rapidrecognitionandpickingpointsautomaticpositioningmethodfortablegrapeinnaturalenvironment |