Fruit Detection and Counting in Apple Orchards Based on Improved Yolov7 and Multi-Object Tracking Methods
With the increasing popularity of online fruit sales, accurately predicting fruit yields has become crucial for optimizing logistics and storage strategies. However, existing manual vision-based systems and sensor methods have proven inadequate for solving the complex problem of fruit yield counting...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-06-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/23/13/5903 |
| _version_ | 1797590856114372608 |
|---|---|
| author | Jing Hu Chuang Fan Zhoupu Wang Jinglin Ruan Suyin Wu |
| author_facet | Jing Hu Chuang Fan Zhoupu Wang Jinglin Ruan Suyin Wu |
| author_sort | Jing Hu |
| collection | DOAJ |
| description | With the increasing popularity of online fruit sales, accurately predicting fruit yields has become crucial for optimizing logistics and storage strategies. However, existing manual vision-based systems and sensor methods have proven inadequate for solving the complex problem of fruit yield counting, as they struggle with issues such as crop overlap and variable lighting conditions. Recently CNN-based object detection models have emerged as a promising solution in the field of computer vision, but their effectiveness is limited in agricultural scenarios due to challenges such as occlusion and dissimilarity among the same fruits. To address this issue, we propose a novel variant model that combines the self-attentive mechanism of Vision Transform, a non-CNN network architecture, with Yolov7, a state-of-the-art object detection model. Our model utilizes two attention mechanisms, CBAM and CA, and is trained and tested on a dataset of apple images. In order to enable fruit counting across video frames in complex environments, we incorporate two multi-objective tracking methods based on Kalman filtering and motion trajectory prediction, namely SORT, and Cascade-SORT. Our results show that the Yolov7-CA model achieved a 91.3% mAP and 0.85 F1 score, representing a 4% improvement in mAP and 0.02 improvement in F1 score compared to using Yolov7 alone. Furthermore, three multi-object tracking methods demonstrated a significant improvement in MAE for inter-frame counting across all three test videos, with an 0.642 improvement over using yolov7 alone achieved using our multi-object tracking method. These findings suggest that our proposed model has the potential to improve fruit yield assessment methods and could have implications for decision-making in the fruit industry. |
| first_indexed | 2024-03-11T01:29:25Z |
| format | Article |
| id | doaj.art-6f4facc3b15e4d7b9181fed818ec9ed5 |
| institution | Directory Open Access Journal |
| issn | 1424-8220 |
| language | English |
| last_indexed | 2024-03-11T01:29:25Z |
| publishDate | 2023-06-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj.art-6f4facc3b15e4d7b9181fed818ec9ed52023-11-18T17:28:28ZengMDPI AGSensors1424-82202023-06-012313590310.3390/s23135903Fruit Detection and Counting in Apple Orchards Based on Improved Yolov7 and Multi-Object Tracking MethodsJing Hu0Chuang Fan1Zhoupu Wang2Jinglin Ruan3Suyin Wu4School of Mathematics and Computer Science, Wuhan Polytechnic University, Wuhan 430024, ChinaSchool of Mathematics and Computer Science, Wuhan Polytechnic University, Wuhan 430024, ChinaSchool of Mathematics and Computer Science, Wuhan Polytechnic University, Wuhan 430024, ChinaSchool of Mathematics and Computer Science, Wuhan Polytechnic University, Wuhan 430024, ChinaSchool of Mathematics and Computer Science, Wuhan Polytechnic University, Wuhan 430024, ChinaWith the increasing popularity of online fruit sales, accurately predicting fruit yields has become crucial for optimizing logistics and storage strategies. However, existing manual vision-based systems and sensor methods have proven inadequate for solving the complex problem of fruit yield counting, as they struggle with issues such as crop overlap and variable lighting conditions. Recently CNN-based object detection models have emerged as a promising solution in the field of computer vision, but their effectiveness is limited in agricultural scenarios due to challenges such as occlusion and dissimilarity among the same fruits. To address this issue, we propose a novel variant model that combines the self-attentive mechanism of Vision Transform, a non-CNN network architecture, with Yolov7, a state-of-the-art object detection model. Our model utilizes two attention mechanisms, CBAM and CA, and is trained and tested on a dataset of apple images. In order to enable fruit counting across video frames in complex environments, we incorporate two multi-objective tracking methods based on Kalman filtering and motion trajectory prediction, namely SORT, and Cascade-SORT. Our results show that the Yolov7-CA model achieved a 91.3% mAP and 0.85 F1 score, representing a 4% improvement in mAP and 0.02 improvement in F1 score compared to using Yolov7 alone. Furthermore, three multi-object tracking methods demonstrated a significant improvement in MAE for inter-frame counting across all three test videos, with an 0.642 improvement over using yolov7 alone achieved using our multi-object tracking method. These findings suggest that our proposed model has the potential to improve fruit yield assessment methods and could have implications for decision-making in the fruit industry.https://www.mdpi.com/1424-8220/23/13/5903yolov7object detectmulti-object tracingself-attention |
| spellingShingle | Jing Hu Chuang Fan Zhoupu Wang Jinglin Ruan Suyin Wu Fruit Detection and Counting in Apple Orchards Based on Improved Yolov7 and Multi-Object Tracking Methods Sensors yolov7 object detect multi-object tracing self-attention |
| title | Fruit Detection and Counting in Apple Orchards Based on Improved Yolov7 and Multi-Object Tracking Methods |
| title_full | Fruit Detection and Counting in Apple Orchards Based on Improved Yolov7 and Multi-Object Tracking Methods |
| title_fullStr | Fruit Detection and Counting in Apple Orchards Based on Improved Yolov7 and Multi-Object Tracking Methods |
| title_full_unstemmed | Fruit Detection and Counting in Apple Orchards Based on Improved Yolov7 and Multi-Object Tracking Methods |
| title_short | Fruit Detection and Counting in Apple Orchards Based on Improved Yolov7 and Multi-Object Tracking Methods |
| title_sort | fruit detection and counting in apple orchards based on improved yolov7 and multi object tracking methods |
| topic | yolov7 object detect multi-object tracing self-attention |
| url | https://www.mdpi.com/1424-8220/23/13/5903 |
| work_keys_str_mv | AT jinghu fruitdetectionandcountinginappleorchardsbasedonimprovedyolov7andmultiobjecttrackingmethods AT chuangfan fruitdetectionandcountinginappleorchardsbasedonimprovedyolov7andmultiobjecttrackingmethods AT zhoupuwang fruitdetectionandcountinginappleorchardsbasedonimprovedyolov7andmultiobjecttrackingmethods AT jinglinruan fruitdetectionandcountinginappleorchardsbasedonimprovedyolov7andmultiobjecttrackingmethods AT suyinwu fruitdetectionandcountinginappleorchardsbasedonimprovedyolov7andmultiobjecttrackingmethods |