Method for Identifying Litchi Picking Position Based on YOLOv5 and PSPNet
China has the largest output of litchi in the world. However, at present, litchi is mainly picked manually, fruit farmers have high labor intensity and low efficiency. This means the intelligent unmanned picking system has broad prospects. The precise location of the main stem picking point of litch...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2022-04-01
|
| Series: | Remote Sensing |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2072-4292/14/9/2004 |
| _version_ | 1797503134543642624 |
|---|---|
| author | Xiaokang Qi Jingshi Dong Yubin Lan Hang Zhu |
| author_facet | Xiaokang Qi Jingshi Dong Yubin Lan Hang Zhu |
| author_sort | Xiaokang Qi |
| collection | DOAJ |
| description | China has the largest output of litchi in the world. However, at present, litchi is mainly picked manually, fruit farmers have high labor intensity and low efficiency. This means the intelligent unmanned picking system has broad prospects. The precise location of the main stem picking point of litchi is very important for the path planning of an unmanned system. Some researchers have identified the fruit and branches of litchi; however, there is relatively little research on the location of the main stem picking point of litchi. So, this paper presents a new open-access workflow for detecting accurate picking locations on the main stems and presents data used in the case study. At the same time, this paper also compares several different network architectures for main stem detection and segmentation and selects YOLOv5 and PSPNet as the most promising models for main stem detection and segmentation tasks, respectively. The workflow combines deep learning and traditional image processing algorithms to calculate the accurate location information of litchi main stem picking points in the litchi image. This workflow takes YOLOv5 as the target detection model to detect the litchi main stem in the litchi image, then extracts the detected region of interest (ROI) of the litchi main stem, uses PSPNet semantic segmentation model to semantically segment the ROI image of the main stem, carries out image post-processing operation on the ROI image of the main stem after semantic segmentation, and obtains the pixel coordinates of picking points in the ROI image of the main stem. After coordinate conversion, the pixel coordinates of the main stem picking points of the original litchi image are obtained, and the picking points are drawn on the litchi image. At present, the workflow can obtain the accurate position information of the main stem picking point in the litchi image. The recall and precision of this method were 76.29% and 92.50%, respectively, which lays a foundation for the subsequent work of obtaining the three-dimensional coordinates of the main stem picking point according to the image depth information, even though we have not done this work in this paper. |
| first_indexed | 2024-03-10T03:46:11Z |
| format | Article |
| id | doaj.art-b571d24ea64744bda78dffbd2d0f3130 |
| institution | Directory Open Access Journal |
| issn | 2072-4292 |
| language | English |
| last_indexed | 2024-03-10T03:46:11Z |
| publishDate | 2022-04-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Remote Sensing |
| spelling | doaj.art-b571d24ea64744bda78dffbd2d0f31302023-11-23T09:09:09ZengMDPI AGRemote Sensing2072-42922022-04-01149200410.3390/rs14092004Method for Identifying Litchi Picking Position Based on YOLOv5 and PSPNetXiaokang Qi0Jingshi Dong1Yubin Lan2Hang Zhu3School of Mechanical and Aerospace Engineering, Jilin University, Changchun 130025, ChinaSchool of Mechanical and Aerospace Engineering, Jilin University, Changchun 130025, ChinaSchool of Agricultural Engineering and Food Science, Shandong University of Technology, Zibo 255000, ChinaSchool of Mechanical and Aerospace Engineering, Jilin University, Changchun 130025, ChinaChina has the largest output of litchi in the world. However, at present, litchi is mainly picked manually, fruit farmers have high labor intensity and low efficiency. This means the intelligent unmanned picking system has broad prospects. The precise location of the main stem picking point of litchi is very important for the path planning of an unmanned system. Some researchers have identified the fruit and branches of litchi; however, there is relatively little research on the location of the main stem picking point of litchi. So, this paper presents a new open-access workflow for detecting accurate picking locations on the main stems and presents data used in the case study. At the same time, this paper also compares several different network architectures for main stem detection and segmentation and selects YOLOv5 and PSPNet as the most promising models for main stem detection and segmentation tasks, respectively. The workflow combines deep learning and traditional image processing algorithms to calculate the accurate location information of litchi main stem picking points in the litchi image. This workflow takes YOLOv5 as the target detection model to detect the litchi main stem in the litchi image, then extracts the detected region of interest (ROI) of the litchi main stem, uses PSPNet semantic segmentation model to semantically segment the ROI image of the main stem, carries out image post-processing operation on the ROI image of the main stem after semantic segmentation, and obtains the pixel coordinates of picking points in the ROI image of the main stem. After coordinate conversion, the pixel coordinates of the main stem picking points of the original litchi image are obtained, and the picking points are drawn on the litchi image. At present, the workflow can obtain the accurate position information of the main stem picking point in the litchi image. The recall and precision of this method were 76.29% and 92.50%, respectively, which lays a foundation for the subsequent work of obtaining the three-dimensional coordinates of the main stem picking point according to the image depth information, even though we have not done this work in this paper.https://www.mdpi.com/2072-4292/14/9/2004YOLOv5PSPNetlitchideep learningpicking pointimage processing |
| spellingShingle | Xiaokang Qi Jingshi Dong Yubin Lan Hang Zhu Method for Identifying Litchi Picking Position Based on YOLOv5 and PSPNet Remote Sensing YOLOv5 PSPNet litchi deep learning picking point image processing |
| title | Method for Identifying Litchi Picking Position Based on YOLOv5 and PSPNet |
| title_full | Method for Identifying Litchi Picking Position Based on YOLOv5 and PSPNet |
| title_fullStr | Method for Identifying Litchi Picking Position Based on YOLOv5 and PSPNet |
| title_full_unstemmed | Method for Identifying Litchi Picking Position Based on YOLOv5 and PSPNet |
| title_short | Method for Identifying Litchi Picking Position Based on YOLOv5 and PSPNet |
| title_sort | method for identifying litchi picking position based on yolov5 and pspnet |
| topic | YOLOv5 PSPNet litchi deep learning picking point image processing |
| url | https://www.mdpi.com/2072-4292/14/9/2004 |
| work_keys_str_mv | AT xiaokangqi methodforidentifyinglitchipickingpositionbasedonyolov5andpspnet AT jingshidong methodforidentifyinglitchipickingpositionbasedonyolov5andpspnet AT yubinlan methodforidentifyinglitchipickingpositionbasedonyolov5andpspnet AT hangzhu methodforidentifyinglitchipickingpositionbasedonyolov5andpspnet |