Instance Segmentation of Lotus Pods and Stalks in Unstructured Planting Environment Based on Improved YOLOv5
Accurate segmentation of lotus pods and stalks with pose variability is a prerequisite for realizing the robotic harvesting of lotus pods. However, the complex growth environment of lotus pods causes great difficulties in conducting the above task. In this study, an instance segmentation model, LPSS...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-08-01
|
| Series: | Agriculture |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2077-0472/13/8/1568 |
| _version_ | 1797585924755816448 |
|---|---|
| author | Ange Lu Lingzhi Ma Hao Cui Jun Liu Qiucheng Ma |
| author_facet | Ange Lu Lingzhi Ma Hao Cui Jun Liu Qiucheng Ma |
| author_sort | Ange Lu |
| collection | DOAJ |
| description | Accurate segmentation of lotus pods and stalks with pose variability is a prerequisite for realizing the robotic harvesting of lotus pods. However, the complex growth environment of lotus pods causes great difficulties in conducting the above task. In this study, an instance segmentation model, LPSS-YOLOv5, for lotus pods and stalks based on the latest YOLOv5 v7.0 instance segmentation model was proposed. The CBAM attention mechanism was integrated into the network to improve the model’s feature extraction ability. The scale distribution of the multi-scale feature layer was adjusted, a 160 × 160 small-scale detection layer was added, and the original 20 × 20 large-scale detection layer was removed, which improved the model’s segmentation accuracy for small-scale lotus stalks and reduced the model size. On the medium-large scale test set, LPSS-YOLOv5 achieved a mask mAP<sub>0</sub>.<sub>5</sub> of 99.3% for all classes. On the small-scale test set, the mAP<sub>0</sub>.<sub>5</sub> for all classes and AP<sub>0</sub>.<sub>5</sub> for stalks were 88.8% and 83.3%, which were 2.6% and 5.0% higher than the baseline, respectively. Compared with the mainstream Mask R-CNN and YOLACT models, LPSS-YOLOv5 showed a much higher segmentation accuracy, speed, and smaller size. The 2D and 3D localization tests verified that LPSS-YOLOv5 could effectively support the picking point localization and the pod–stalk affiliation confirmation. |
| first_indexed | 2024-03-11T00:13:03Z |
| format | Article |
| id | doaj.art-f81574a4a01547b09c02c44c8e815286 |
| institution | Directory Open Access Journal |
| issn | 2077-0472 |
| language | English |
| last_indexed | 2024-03-11T00:13:03Z |
| publishDate | 2023-08-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Agriculture |
| spelling | doaj.art-f81574a4a01547b09c02c44c8e8152862023-11-18T23:51:59ZengMDPI AGAgriculture2077-04722023-08-01138156810.3390/agriculture13081568Instance Segmentation of Lotus Pods and Stalks in Unstructured Planting Environment Based on Improved YOLOv5Ange Lu0Lingzhi Ma1Hao Cui2Jun Liu3Qiucheng Ma4School of Mechanical Engineering and Mechanics, Xiangtan University, Xiangtan 411105, ChinaSchool of Mechanical Engineering and Mechanics, Xiangtan University, Xiangtan 411105, ChinaSchool of Mechanical Engineering and Mechanics, Xiangtan University, Xiangtan 411105, ChinaSchool of Mechanical Engineering and Mechanics, Xiangtan University, Xiangtan 411105, ChinaSchool of Mechanical Engineering and Mechanics, Xiangtan University, Xiangtan 411105, ChinaAccurate segmentation of lotus pods and stalks with pose variability is a prerequisite for realizing the robotic harvesting of lotus pods. However, the complex growth environment of lotus pods causes great difficulties in conducting the above task. In this study, an instance segmentation model, LPSS-YOLOv5, for lotus pods and stalks based on the latest YOLOv5 v7.0 instance segmentation model was proposed. The CBAM attention mechanism was integrated into the network to improve the model’s feature extraction ability. The scale distribution of the multi-scale feature layer was adjusted, a 160 × 160 small-scale detection layer was added, and the original 20 × 20 large-scale detection layer was removed, which improved the model’s segmentation accuracy for small-scale lotus stalks and reduced the model size. On the medium-large scale test set, LPSS-YOLOv5 achieved a mask mAP<sub>0</sub>.<sub>5</sub> of 99.3% for all classes. On the small-scale test set, the mAP<sub>0</sub>.<sub>5</sub> for all classes and AP<sub>0</sub>.<sub>5</sub> for stalks were 88.8% and 83.3%, which were 2.6% and 5.0% higher than the baseline, respectively. Compared with the mainstream Mask R-CNN and YOLACT models, LPSS-YOLOv5 showed a much higher segmentation accuracy, speed, and smaller size. The 2D and 3D localization tests verified that LPSS-YOLOv5 could effectively support the picking point localization and the pod–stalk affiliation confirmation.https://www.mdpi.com/2077-0472/13/8/1568lotus podsinstance segmentationdeep learningYOLOv5attention mechanism |
| spellingShingle | Ange Lu Lingzhi Ma Hao Cui Jun Liu Qiucheng Ma Instance Segmentation of Lotus Pods and Stalks in Unstructured Planting Environment Based on Improved YOLOv5 Agriculture lotus pods instance segmentation deep learning YOLOv5 attention mechanism |
| title | Instance Segmentation of Lotus Pods and Stalks in Unstructured Planting Environment Based on Improved YOLOv5 |
| title_full | Instance Segmentation of Lotus Pods and Stalks in Unstructured Planting Environment Based on Improved YOLOv5 |
| title_fullStr | Instance Segmentation of Lotus Pods and Stalks in Unstructured Planting Environment Based on Improved YOLOv5 |
| title_full_unstemmed | Instance Segmentation of Lotus Pods and Stalks in Unstructured Planting Environment Based on Improved YOLOv5 |
| title_short | Instance Segmentation of Lotus Pods and Stalks in Unstructured Planting Environment Based on Improved YOLOv5 |
| title_sort | instance segmentation of lotus pods and stalks in unstructured planting environment based on improved yolov5 |
| topic | lotus pods instance segmentation deep learning YOLOv5 attention mechanism |
| url | https://www.mdpi.com/2077-0472/13/8/1568 |
| work_keys_str_mv | AT angelu instancesegmentationoflotuspodsandstalksinunstructuredplantingenvironmentbasedonimprovedyolov5 AT lingzhima instancesegmentationoflotuspodsandstalksinunstructuredplantingenvironmentbasedonimprovedyolov5 AT haocui instancesegmentationoflotuspodsandstalksinunstructuredplantingenvironmentbasedonimprovedyolov5 AT junliu instancesegmentationoflotuspodsandstalksinunstructuredplantingenvironmentbasedonimprovedyolov5 AT qiuchengma instancesegmentationoflotuspodsandstalksinunstructuredplantingenvironmentbasedonimprovedyolov5 |