Vision-Based Moving Obstacle Detection and Tracking in Paddy Field Using Improved Yolov3 and Deep SORT
Using intelligent agricultural machines in paddy fields has received great attention. An obstacle avoidance system is required with the development of agricultural machines. In order to make the machines more intelligent, detecting and tracking obstacles, especially the moving obstacles in paddy fie...
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2020-07-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/20/15/4082 |
| _version_ | 1797561638375653376 |
|---|---|
| author | Zhengjun Qiu Nan Zhao Lei Zhou Mengcen Wang Liangliang Yang Hui Fang Yong He Yufei Liu |
| author_facet | Zhengjun Qiu Nan Zhao Lei Zhou Mengcen Wang Liangliang Yang Hui Fang Yong He Yufei Liu |
| author_sort | Zhengjun Qiu |
| collection | DOAJ |
| description | Using intelligent agricultural machines in paddy fields has received great attention. An obstacle avoidance system is required with the development of agricultural machines. In order to make the machines more intelligent, detecting and tracking obstacles, especially the moving obstacles in paddy fields, is the basis of obstacle avoidance. To achieve this goal, a red, green and blue (RGB) camera and a computer were used to build a machine vision system, mounted on a transplanter. A method that combined the improved You Only Look Once version 3 (Yolov3) and deep Simple Online and Realtime Tracking (deep SORT) was used to detect and track typical moving obstacles, and figure out the center point positions of the obstacles in paddy fields. The improved Yolov3 has 23 residual blocks and upsamples only once, and has new loss calculation functions. Results showed that the improved Yolov3 obtained mean intersection over union (mIoU) score of 0.779 and was 27.3% faster in processing speed than standard Yolov3 on a self-created test dataset of moving obstacles (human and water buffalo) in paddy fields. An acceptable performance for detecting and tracking could be obtained in a real paddy field test with an average processing speed of 5–7 frames per second (FPS), which satisfies actual work demands. In future research, the proposed system could support the intelligent agriculture machines more flexible in autonomous navigation. |
| first_indexed | 2024-03-10T18:17:45Z |
| format | Article |
| id | doaj.art-3e2ba5a7b59f472abbad405dd2b8e6a4 |
| institution | Directory Open Access Journal |
| issn | 1424-8220 |
| language | English |
| last_indexed | 2024-03-10T18:17:45Z |
| publishDate | 2020-07-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj.art-3e2ba5a7b59f472abbad405dd2b8e6a42023-11-20T07:35:16ZengMDPI AGSensors1424-82202020-07-012015408210.3390/s20154082Vision-Based Moving Obstacle Detection and Tracking in Paddy Field Using Improved Yolov3 and Deep SORTZhengjun Qiu0Nan Zhao1Lei Zhou2Mengcen Wang3Liangliang Yang4Hui Fang5Yong He6Yufei Liu7College of Biosystems Engineering and Food Science, Zhejiang University, Hangzhou 310058, ChinaCollege of Biosystems Engineering and Food Science, Zhejiang University, Hangzhou 310058, ChinaCollege of Biosystems Engineering and Food Science, Zhejiang University, Hangzhou 310058, ChinaMinistry of Agriculture Key Laboratory of Molecular Biology of Crop Pathogens and Insects, Institute of Pesticide and Environmental Toxicology, Zhejiang University, Hangzhou 310058, ChinaFaculty of Engineering, Kitami Institute of Technology, Koen-cho 165, Kitami, Hokkaido 090-8507, JapanCollege of Biosystems Engineering and Food Science, Zhejiang University, Hangzhou 310058, ChinaCollege of Biosystems Engineering and Food Science, Zhejiang University, Hangzhou 310058, ChinaCollege of Biosystems Engineering and Food Science, Zhejiang University, Hangzhou 310058, ChinaUsing intelligent agricultural machines in paddy fields has received great attention. An obstacle avoidance system is required with the development of agricultural machines. In order to make the machines more intelligent, detecting and tracking obstacles, especially the moving obstacles in paddy fields, is the basis of obstacle avoidance. To achieve this goal, a red, green and blue (RGB) camera and a computer were used to build a machine vision system, mounted on a transplanter. A method that combined the improved You Only Look Once version 3 (Yolov3) and deep Simple Online and Realtime Tracking (deep SORT) was used to detect and track typical moving obstacles, and figure out the center point positions of the obstacles in paddy fields. The improved Yolov3 has 23 residual blocks and upsamples only once, and has new loss calculation functions. Results showed that the improved Yolov3 obtained mean intersection over union (mIoU) score of 0.779 and was 27.3% faster in processing speed than standard Yolov3 on a self-created test dataset of moving obstacles (human and water buffalo) in paddy fields. An acceptable performance for detecting and tracking could be obtained in a real paddy field test with an average processing speed of 5–7 frames per second (FPS), which satisfies actual work demands. In future research, the proposed system could support the intelligent agriculture machines more flexible in autonomous navigation.https://www.mdpi.com/1424-8220/20/15/4082machine visiondeep learningdetecting and trackingmoving obstaclespaddy field |
| spellingShingle | Zhengjun Qiu Nan Zhao Lei Zhou Mengcen Wang Liangliang Yang Hui Fang Yong He Yufei Liu Vision-Based Moving Obstacle Detection and Tracking in Paddy Field Using Improved Yolov3 and Deep SORT Sensors machine vision deep learning detecting and tracking moving obstacles paddy field |
| title | Vision-Based Moving Obstacle Detection and Tracking in Paddy Field Using Improved Yolov3 and Deep SORT |
| title_full | Vision-Based Moving Obstacle Detection and Tracking in Paddy Field Using Improved Yolov3 and Deep SORT |
| title_fullStr | Vision-Based Moving Obstacle Detection and Tracking in Paddy Field Using Improved Yolov3 and Deep SORT |
| title_full_unstemmed | Vision-Based Moving Obstacle Detection and Tracking in Paddy Field Using Improved Yolov3 and Deep SORT |
| title_short | Vision-Based Moving Obstacle Detection and Tracking in Paddy Field Using Improved Yolov3 and Deep SORT |
| title_sort | vision based moving obstacle detection and tracking in paddy field using improved yolov3 and deep sort |
| topic | machine vision deep learning detecting and tracking moving obstacles paddy field |
| url | https://www.mdpi.com/1424-8220/20/15/4082 |
| work_keys_str_mv | AT zhengjunqiu visionbasedmovingobstacledetectionandtrackinginpaddyfieldusingimprovedyolov3anddeepsort AT nanzhao visionbasedmovingobstacledetectionandtrackinginpaddyfieldusingimprovedyolov3anddeepsort AT leizhou visionbasedmovingobstacledetectionandtrackinginpaddyfieldusingimprovedyolov3anddeepsort AT mengcenwang visionbasedmovingobstacledetectionandtrackinginpaddyfieldusingimprovedyolov3anddeepsort AT liangliangyang visionbasedmovingobstacledetectionandtrackinginpaddyfieldusingimprovedyolov3anddeepsort AT huifang visionbasedmovingobstacledetectionandtrackinginpaddyfieldusingimprovedyolov3anddeepsort AT yonghe visionbasedmovingobstacledetectionandtrackinginpaddyfieldusingimprovedyolov3anddeepsort AT yufeiliu visionbasedmovingobstacledetectionandtrackinginpaddyfieldusingimprovedyolov3anddeepsort |