“Is this blueberry ripe?”: a blueberry ripeness detection algorithm for use on picking robots
Blueberries are grown worldwide because of their high nutritional value; however, manual picking is difficult, and expert pickers are scarce. To meet the real needs of the market, picking robots that can identify the ripeness of blueberries are increasingly being used to replace manual operators. Ho...
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Frontiers Media S.A.
2023-06-01
|
| Series: | Frontiers in Plant Science |
| Subjects: | |
| Online Access: | https://www.frontiersin.org/articles/10.3389/fpls.2023.1198650/full |
| _version_ | 1797808334052524032 |
|---|---|
| author | Yan Liu Hongtao Zheng Yonghua Zhang Qiujie Zhang Hongli Chen Hongli Chen Xueyong Xu Gaoyang Wang |
| author_facet | Yan Liu Hongtao Zheng Yonghua Zhang Qiujie Zhang Hongli Chen Hongli Chen Xueyong Xu Gaoyang Wang |
| author_sort | Yan Liu |
| collection | DOAJ |
| description | Blueberries are grown worldwide because of their high nutritional value; however, manual picking is difficult, and expert pickers are scarce. To meet the real needs of the market, picking robots that can identify the ripeness of blueberries are increasingly being used to replace manual operators. However, they struggle to accurately identify the ripeness of blueberries because of the heavy shading between the fruits and the small size of the fruit. This makes it difficult to obtain sufficient information on characteristics; and the disturbances caused by environmental changes remain unsolved. Additionally, the picking robot has limited computational power for running complex algorithms. To address these issues, we propose a new YOLO-based algorithm to detect the ripeness of blueberry fruits. The algorithm improves the structure of YOLOv5x. We replaced the fully connected layer with a one-dimensional convolution and also replaced the high-latitude convolution with a null convolution based on the structure of CBAM, and finally obtained a lightweight CBAM structure with efficient attention-guiding capability (Little-CBAM), which we embedded into MobileNetv3 while replacing the original backbone structure with the improved MobileNetv3. We expanded the original three-layer neck path by one to create a larger-scale detection layer leading from the backbone network. We added a multi-scale fusion module to the channel attention mechanism to build a multi-method feature extractor (MSSENet) and then embedded the designed channel attention module into the head network, which can significantly enhance the feature representation capability of the small target detection network and the anti-interference capability of the algorithm. Considering that these improvements will significantly extend the training time of the algorithm, we used EIOU_Loss instead of CIOU_Loss, whereas the k-means++ algorithm was used to cluster the detection frames such that the generated predefined anchor frames are better adapted to the scale of the blueberries. The algorithm in this study achieved a final mAP of 78.3% on the PC terminal, which was 9% higher than that of YOLOv5x, and the FPS was 2.1 times higher than that of YOLOv5x. By translating the algorithm into a picking robot, the algorithm in this study ran at 47 FPS and achieved real-time detection well beyond that achieved manually. |
| first_indexed | 2024-03-13T06:36:00Z |
| format | Article |
| id | doaj.art-69b00b77d9714db984dd7f284a77ce2a |
| institution | Directory Open Access Journal |
| issn | 1664-462X |
| language | English |
| last_indexed | 2024-03-13T06:36:00Z |
| publishDate | 2023-06-01 |
| publisher | Frontiers Media S.A. |
| record_format | Article |
| series | Frontiers in Plant Science |
| spelling | doaj.art-69b00b77d9714db984dd7f284a77ce2a2023-06-09T05:13:16ZengFrontiers Media S.A.Frontiers in Plant Science1664-462X2023-06-011410.3389/fpls.2023.11986501198650“Is this blueberry ripe?”: a blueberry ripeness detection algorithm for use on picking robotsYan Liu0Hongtao Zheng1Yonghua Zhang2Qiujie Zhang3Hongli Chen4Hongli Chen5Xueyong Xu6Gaoyang Wang7School of Information and Electrical Engineering, Hangzhou City University, Hangzhou, ChinaSchool of Information and Electrical Engineering, Hangzhou City University, Hangzhou, ChinaSchool of Information and Electrical Engineering, Hangzhou City University, Hangzhou, ChinaSchool of Information and Electrical Engineering, Hangzhou City University, Hangzhou, ChinaSchool of Information and Electrical Engineering, Hangzhou City University, Hangzhou, ChinaSchool of Information and Electronic Engineering, Zhejiang University, Hangzhou, ChinaSchool of Computer Science and Technology, Zhejiang Sci-Tech University, Hangzhou, ChinaSchool of Control Science and Engineering, Zhejiang University, Hangzhou, ChinaBlueberries are grown worldwide because of their high nutritional value; however, manual picking is difficult, and expert pickers are scarce. To meet the real needs of the market, picking robots that can identify the ripeness of blueberries are increasingly being used to replace manual operators. However, they struggle to accurately identify the ripeness of blueberries because of the heavy shading between the fruits and the small size of the fruit. This makes it difficult to obtain sufficient information on characteristics; and the disturbances caused by environmental changes remain unsolved. Additionally, the picking robot has limited computational power for running complex algorithms. To address these issues, we propose a new YOLO-based algorithm to detect the ripeness of blueberry fruits. The algorithm improves the structure of YOLOv5x. We replaced the fully connected layer with a one-dimensional convolution and also replaced the high-latitude convolution with a null convolution based on the structure of CBAM, and finally obtained a lightweight CBAM structure with efficient attention-guiding capability (Little-CBAM), which we embedded into MobileNetv3 while replacing the original backbone structure with the improved MobileNetv3. We expanded the original three-layer neck path by one to create a larger-scale detection layer leading from the backbone network. We added a multi-scale fusion module to the channel attention mechanism to build a multi-method feature extractor (MSSENet) and then embedded the designed channel attention module into the head network, which can significantly enhance the feature representation capability of the small target detection network and the anti-interference capability of the algorithm. Considering that these improvements will significantly extend the training time of the algorithm, we used EIOU_Loss instead of CIOU_Loss, whereas the k-means++ algorithm was used to cluster the detection frames such that the generated predefined anchor frames are better adapted to the scale of the blueberries. The algorithm in this study achieved a final mAP of 78.3% on the PC terminal, which was 9% higher than that of YOLOv5x, and the FPS was 2.1 times higher than that of YOLOv5x. By translating the algorithm into a picking robot, the algorithm in this study ran at 47 FPS and achieved real-time detection well beyond that achieved manually.https://www.frontiersin.org/articles/10.3389/fpls.2023.1198650/fullvegetableblueberriesYOLOv5xMobileNetV3Little-CBAMMSSENet |
| spellingShingle | Yan Liu Hongtao Zheng Yonghua Zhang Qiujie Zhang Hongli Chen Hongli Chen Xueyong Xu Gaoyang Wang “Is this blueberry ripe?”: a blueberry ripeness detection algorithm for use on picking robots Frontiers in Plant Science vegetable blueberries YOLOv5x MobileNetV3 Little-CBAM MSSENet |
| title | “Is this blueberry ripe?”: a blueberry ripeness detection algorithm for use on picking robots |
| title_full | “Is this blueberry ripe?”: a blueberry ripeness detection algorithm for use on picking robots |
| title_fullStr | “Is this blueberry ripe?”: a blueberry ripeness detection algorithm for use on picking robots |
| title_full_unstemmed | “Is this blueberry ripe?”: a blueberry ripeness detection algorithm for use on picking robots |
| title_short | “Is this blueberry ripe?”: a blueberry ripeness detection algorithm for use on picking robots |
| title_sort | is this blueberry ripe a blueberry ripeness detection algorithm for use on picking robots |
| topic | vegetable blueberries YOLOv5x MobileNetV3 Little-CBAM MSSENet |
| url | https://www.frontiersin.org/articles/10.3389/fpls.2023.1198650/full |
| work_keys_str_mv | AT yanliu isthisblueberryripeablueberryripenessdetectionalgorithmforuseonpickingrobots AT hongtaozheng isthisblueberryripeablueberryripenessdetectionalgorithmforuseonpickingrobots AT yonghuazhang isthisblueberryripeablueberryripenessdetectionalgorithmforuseonpickingrobots AT qiujiezhang isthisblueberryripeablueberryripenessdetectionalgorithmforuseonpickingrobots AT honglichen isthisblueberryripeablueberryripenessdetectionalgorithmforuseonpickingrobots AT honglichen isthisblueberryripeablueberryripenessdetectionalgorithmforuseonpickingrobots AT xueyongxu isthisblueberryripeablueberryripenessdetectionalgorithmforuseonpickingrobots AT gaoyangwang isthisblueberryripeablueberryripenessdetectionalgorithmforuseonpickingrobots |