Adaptive Active Positioning of <i>Camellia oleifera</i> Fruit Picking Points: Classical Image Processing and YOLOv7 Fusion Algorithm
<i>Camellia oleifera</i> fruits are randomly distributed in an orchard, and the fruits are easily blocked or covered by leaves. In addition, the colors of leaves and fruits are alike, and flowers and fruits grow at the same time, presenting many ambiguities. The large shock force will ca...
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2022-12-01
|
| Series: | Applied Sciences |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2076-3417/12/24/12959 |
| _version_ | 1797461540989829120 |
|---|---|
| author | Yunhe Zhou Yunchao Tang Xiangjun Zou Mingliang Wu Wei Tang Fan Meng Yunqi Zhang Hanwen Kang |
| author_facet | Yunhe Zhou Yunchao Tang Xiangjun Zou Mingliang Wu Wei Tang Fan Meng Yunqi Zhang Hanwen Kang |
| author_sort | Yunhe Zhou |
| collection | DOAJ |
| description | <i>Camellia oleifera</i> fruits are randomly distributed in an orchard, and the fruits are easily blocked or covered by leaves. In addition, the colors of leaves and fruits are alike, and flowers and fruits grow at the same time, presenting many ambiguities. The large shock force will cause flowers to fall and affect the yield. As a result, accurate positioning becomes a difficult problem for robot picking. Therefore, studying target recognition and localization of <i>Camellia oleifera</i> fruits in complex environments has many difficulties. In this paper, a fusion method of deep learning based on visual perception and image processing is proposed to adaptively and actively locate fruit recognition and picking points for <i>Camellia oleifera</i> fruits. First, to adapt to the target classification and recognition of complex scenes in the field, the parameters of the You Only Live Once v7 (YOLOv7) model were optimized and selected to achieve <i>Camellia oleifera</i> fruits’ detection and determine the center point of the fruit recognition frame. Then, image processing and a geometric algorithm are used to process the image, segment, and determine the morphology of the fruit, extract the centroid of the outline of <i>Camellia oleifera</i> fruit, and then analyze the position deviation of its centroid point and the center point in the YOLO recognition frame. The frontlighting, backlight, partial occlusion, and other test conditions for the perceptual recognition processing were validated with several experiments. The results demonstrate that the <i>precision</i> of YOLOv7 is close to that of YOLOv5s, and the <i>mean average precision</i> of YOLOv7 is higher than that of YOLOv5s. For some occluded <i>Camellia oleifera</i> fruits, the YOLOv7 algorithm is better than the YOLOv5s algorithm, which improves the detection accuracy of <i>Camellia oleifera</i> fruits. The contour of <i>Camellia oleifera</i> fruits can be extracted entirely via image processing. The average position deviation between the centroid point of the image extraction and the center point of the YOLO recognition frame is 2.86 pixels; thus, the center point of the YOLO recognition frame is approximately considered to be consistent with the centroid point of the image extraction. |
| first_indexed | 2024-03-09T17:20:52Z |
| format | Article |
| id | doaj.art-2a3beb6db7a644baa657ed558a3d848f |
| institution | Directory Open Access Journal |
| issn | 2076-3417 |
| language | English |
| last_indexed | 2024-03-09T17:20:52Z |
| publishDate | 2022-12-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Applied Sciences |
| spelling | doaj.art-2a3beb6db7a644baa657ed558a3d848f2023-11-24T13:07:42ZengMDPI AGApplied Sciences2076-34172022-12-0112241295910.3390/app122412959Adaptive Active Positioning of <i>Camellia oleifera</i> Fruit Picking Points: Classical Image Processing and YOLOv7 Fusion AlgorithmYunhe Zhou0Yunchao Tang1Xiangjun Zou2Mingliang Wu3Wei Tang4Fan Meng5Yunqi Zhang6Hanwen Kang7College of Urban and Rural Construction, Zhongkai University of Agriculture and Engineering, Guangzhou 510225, ChinaCollege of Urban and Rural Construction, Zhongkai University of Agriculture and Engineering, Guangzhou 510225, ChinaFoshan-Zhongke Innovation Research Institute of Intelligent Agriculture and Robotics, Foshan 528000, ChinaCollege of Mechanical and Electrical Engineering, Hunan Agricultural University, Changsha 410128, ChinaFoshan-Zhongke Innovation Research Institute of Intelligent Agriculture and Robotics, Foshan 528000, ChinaCollege of Urban and Rural Construction, Zhongkai University of Agriculture and Engineering, Guangzhou 510225, ChinaCollege of Urban and Rural Construction, Zhongkai University of Agriculture and Engineering, Guangzhou 510225, ChinaCollege of Engineering, South China Agricultural University, Guangzhou 510642, China<i>Camellia oleifera</i> fruits are randomly distributed in an orchard, and the fruits are easily blocked or covered by leaves. In addition, the colors of leaves and fruits are alike, and flowers and fruits grow at the same time, presenting many ambiguities. The large shock force will cause flowers to fall and affect the yield. As a result, accurate positioning becomes a difficult problem for robot picking. Therefore, studying target recognition and localization of <i>Camellia oleifera</i> fruits in complex environments has many difficulties. In this paper, a fusion method of deep learning based on visual perception and image processing is proposed to adaptively and actively locate fruit recognition and picking points for <i>Camellia oleifera</i> fruits. First, to adapt to the target classification and recognition of complex scenes in the field, the parameters of the You Only Live Once v7 (YOLOv7) model were optimized and selected to achieve <i>Camellia oleifera</i> fruits’ detection and determine the center point of the fruit recognition frame. Then, image processing and a geometric algorithm are used to process the image, segment, and determine the morphology of the fruit, extract the centroid of the outline of <i>Camellia oleifera</i> fruit, and then analyze the position deviation of its centroid point and the center point in the YOLO recognition frame. The frontlighting, backlight, partial occlusion, and other test conditions for the perceptual recognition processing were validated with several experiments. The results demonstrate that the <i>precision</i> of YOLOv7 is close to that of YOLOv5s, and the <i>mean average precision</i> of YOLOv7 is higher than that of YOLOv5s. For some occluded <i>Camellia oleifera</i> fruits, the YOLOv7 algorithm is better than the YOLOv5s algorithm, which improves the detection accuracy of <i>Camellia oleifera</i> fruits. The contour of <i>Camellia oleifera</i> fruits can be extracted entirely via image processing. The average position deviation between the centroid point of the image extraction and the center point of the YOLO recognition frame is 2.86 pixels; thus, the center point of the YOLO recognition frame is approximately considered to be consistent with the centroid point of the image extraction.https://www.mdpi.com/2076-3417/12/24/12959machine visionYOLOv7image processinggrabbing point positioning |
| spellingShingle | Yunhe Zhou Yunchao Tang Xiangjun Zou Mingliang Wu Wei Tang Fan Meng Yunqi Zhang Hanwen Kang Adaptive Active Positioning of <i>Camellia oleifera</i> Fruit Picking Points: Classical Image Processing and YOLOv7 Fusion Algorithm Applied Sciences machine vision YOLOv7 image processing grabbing point positioning |
| title | Adaptive Active Positioning of <i>Camellia oleifera</i> Fruit Picking Points: Classical Image Processing and YOLOv7 Fusion Algorithm |
| title_full | Adaptive Active Positioning of <i>Camellia oleifera</i> Fruit Picking Points: Classical Image Processing and YOLOv7 Fusion Algorithm |
| title_fullStr | Adaptive Active Positioning of <i>Camellia oleifera</i> Fruit Picking Points: Classical Image Processing and YOLOv7 Fusion Algorithm |
| title_full_unstemmed | Adaptive Active Positioning of <i>Camellia oleifera</i> Fruit Picking Points: Classical Image Processing and YOLOv7 Fusion Algorithm |
| title_short | Adaptive Active Positioning of <i>Camellia oleifera</i> Fruit Picking Points: Classical Image Processing and YOLOv7 Fusion Algorithm |
| title_sort | adaptive active positioning of i camellia oleifera i fruit picking points classical image processing and yolov7 fusion algorithm |
| topic | machine vision YOLOv7 image processing grabbing point positioning |
| url | https://www.mdpi.com/2076-3417/12/24/12959 |
| work_keys_str_mv | AT yunhezhou adaptiveactivepositioningoficamelliaoleiferaifruitpickingpointsclassicalimageprocessingandyolov7fusionalgorithm AT yunchaotang adaptiveactivepositioningoficamelliaoleiferaifruitpickingpointsclassicalimageprocessingandyolov7fusionalgorithm AT xiangjunzou adaptiveactivepositioningoficamelliaoleiferaifruitpickingpointsclassicalimageprocessingandyolov7fusionalgorithm AT mingliangwu adaptiveactivepositioningoficamelliaoleiferaifruitpickingpointsclassicalimageprocessingandyolov7fusionalgorithm AT weitang adaptiveactivepositioningoficamelliaoleiferaifruitpickingpointsclassicalimageprocessingandyolov7fusionalgorithm AT fanmeng adaptiveactivepositioningoficamelliaoleiferaifruitpickingpointsclassicalimageprocessingandyolov7fusionalgorithm AT yunqizhang adaptiveactivepositioningoficamelliaoleiferaifruitpickingpointsclassicalimageprocessingandyolov7fusionalgorithm AT hanwenkang adaptiveactivepositioningoficamelliaoleiferaifruitpickingpointsclassicalimageprocessingandyolov7fusionalgorithm |