A Deep Learning-Based Vision System Combining Detection and Tracking for Fast On-Line Citrus Sorting
Defective citrus fruits are manually sorted at the moment, which is a time-consuming and cost-expensive process with unsatisfactory accuracy. In this paper, we introduce a deep learning-based vision system implemented on a citrus processing line for fast on-line sorting. For the citrus fruits rotati...
Main Authors: | , , , , |
---|---|
Format: | Article |
Language: | English |
Published: |
Frontiers Media S.A.
2021-02-01
|
Series: | Frontiers in Plant Science |
Subjects: | |
Online Access: | https://www.frontiersin.org/articles/10.3389/fpls.2021.622062/full |
_version_ | 1818677344027541504 |
---|---|
author | Yaohui Chen Yaohui Chen Yaohui Chen Xiaosong An Shumin Gao Shanjun Li Shanjun Li Shanjun Li Shanjun Li Shanjun Li Hanwen Kang |
author_facet | Yaohui Chen Yaohui Chen Yaohui Chen Xiaosong An Shumin Gao Shanjun Li Shanjun Li Shanjun Li Shanjun Li Shanjun Li Hanwen Kang |
author_sort | Yaohui Chen |
collection | DOAJ |
description | Defective citrus fruits are manually sorted at the moment, which is a time-consuming and cost-expensive process with unsatisfactory accuracy. In this paper, we introduce a deep learning-based vision system implemented on a citrus processing line for fast on-line sorting. For the citrus fruits rotating randomly on the conveyor, a convolutional neural network-based detector was developed to detect and temporarily classify the defective ones, and a SORT algorithm-based tracker was adopted to record the classification information along their paths. The true categories of the citrus fruits were identified through the tracked historical information, resulting in high detection precision of 93.6%. Moreover, the linear Kalman filter model was applied to predict the future path of the fruits, which can be used to guide the robot arms to pick out the defective ones. Ultimately, this research presents a practical solution to realize on-line citrus sorting featuring low costs, high efficiency, and accuracy. |
first_indexed | 2024-12-17T08:57:52Z |
format | Article |
id | doaj.art-5de64c770df24ff19c32185a5815d484 |
institution | Directory Open Access Journal |
issn | 1664-462X |
language | English |
last_indexed | 2024-12-17T08:57:52Z |
publishDate | 2021-02-01 |
publisher | Frontiers Media S.A. |
record_format | Article |
series | Frontiers in Plant Science |
spelling | doaj.art-5de64c770df24ff19c32185a5815d4842022-12-21T21:55:52ZengFrontiers Media S.A.Frontiers in Plant Science1664-462X2021-02-011210.3389/fpls.2021.622062622062A Deep Learning-Based Vision System Combining Detection and Tracking for Fast On-Line Citrus SortingYaohui Chen0Yaohui Chen1Yaohui Chen2Xiaosong An3Shumin Gao4Shanjun Li5Shanjun Li6Shanjun Li7Shanjun Li8Shanjun Li9Hanwen Kang10College of Engineering, Huazhong Agricultural University, Wuhan, ChinaKey Laboratory of Agricultural Equipment in Mid-Lower Yangtze River, Ministry of Agriculture and Rural Affairs, Wuhan, ChinaCitrus Mechanization Research Base, Ministry of Agriculture and Rural Affairs, Wuhan, ChinaCollege of Engineering, Huazhong Agricultural University, Wuhan, ChinaCollege of Engineering, Huazhong Agricultural University, Wuhan, ChinaCollege of Engineering, Huazhong Agricultural University, Wuhan, ChinaKey Laboratory of Agricultural Equipment in Mid-Lower Yangtze River, Ministry of Agriculture and Rural Affairs, Wuhan, ChinaCitrus Mechanization Research Base, Ministry of Agriculture and Rural Affairs, Wuhan, ChinaChina Agriculture (Citrus) Research System, Wuhan, ChinaNational R&D Center for Citrus Preservation, Wuhan, ChinaDepartment of Mechanical and Aerospace Engineering, College of Engineering, Monash University, Clayton, VIC, AustraliaDefective citrus fruits are manually sorted at the moment, which is a time-consuming and cost-expensive process with unsatisfactory accuracy. In this paper, we introduce a deep learning-based vision system implemented on a citrus processing line for fast on-line sorting. For the citrus fruits rotating randomly on the conveyor, a convolutional neural network-based detector was developed to detect and temporarily classify the defective ones, and a SORT algorithm-based tracker was adopted to record the classification information along their paths. The true categories of the citrus fruits were identified through the tracked historical information, resulting in high detection precision of 93.6%. Moreover, the linear Kalman filter model was applied to predict the future path of the fruits, which can be used to guide the robot arms to pick out the defective ones. Ultimately, this research presents a practical solution to realize on-line citrus sorting featuring low costs, high efficiency, and accuracy.https://www.frontiersin.org/articles/10.3389/fpls.2021.622062/fulldefective citrus sortingCNN-based detectorSORT-based trackerdeep learningvision system |
spellingShingle | Yaohui Chen Yaohui Chen Yaohui Chen Xiaosong An Shumin Gao Shanjun Li Shanjun Li Shanjun Li Shanjun Li Shanjun Li Hanwen Kang A Deep Learning-Based Vision System Combining Detection and Tracking for Fast On-Line Citrus Sorting Frontiers in Plant Science defective citrus sorting CNN-based detector SORT-based tracker deep learning vision system |
title | A Deep Learning-Based Vision System Combining Detection and Tracking for Fast On-Line Citrus Sorting |
title_full | A Deep Learning-Based Vision System Combining Detection and Tracking for Fast On-Line Citrus Sorting |
title_fullStr | A Deep Learning-Based Vision System Combining Detection and Tracking for Fast On-Line Citrus Sorting |
title_full_unstemmed | A Deep Learning-Based Vision System Combining Detection and Tracking for Fast On-Line Citrus Sorting |
title_short | A Deep Learning-Based Vision System Combining Detection and Tracking for Fast On-Line Citrus Sorting |
title_sort | deep learning based vision system combining detection and tracking for fast on line citrus sorting |
topic | defective citrus sorting CNN-based detector SORT-based tracker deep learning vision system |
url | https://www.frontiersin.org/articles/10.3389/fpls.2021.622062/full |
work_keys_str_mv | AT yaohuichen adeeplearningbasedvisionsystemcombiningdetectionandtrackingforfastonlinecitrussorting AT yaohuichen adeeplearningbasedvisionsystemcombiningdetectionandtrackingforfastonlinecitrussorting AT yaohuichen adeeplearningbasedvisionsystemcombiningdetectionandtrackingforfastonlinecitrussorting AT xiaosongan adeeplearningbasedvisionsystemcombiningdetectionandtrackingforfastonlinecitrussorting AT shumingao adeeplearningbasedvisionsystemcombiningdetectionandtrackingforfastonlinecitrussorting AT shanjunli adeeplearningbasedvisionsystemcombiningdetectionandtrackingforfastonlinecitrussorting AT shanjunli adeeplearningbasedvisionsystemcombiningdetectionandtrackingforfastonlinecitrussorting AT shanjunli adeeplearningbasedvisionsystemcombiningdetectionandtrackingforfastonlinecitrussorting AT shanjunli adeeplearningbasedvisionsystemcombiningdetectionandtrackingforfastonlinecitrussorting AT shanjunli adeeplearningbasedvisionsystemcombiningdetectionandtrackingforfastonlinecitrussorting AT hanwenkang adeeplearningbasedvisionsystemcombiningdetectionandtrackingforfastonlinecitrussorting AT yaohuichen deeplearningbasedvisionsystemcombiningdetectionandtrackingforfastonlinecitrussorting AT yaohuichen deeplearningbasedvisionsystemcombiningdetectionandtrackingforfastonlinecitrussorting AT yaohuichen deeplearningbasedvisionsystemcombiningdetectionandtrackingforfastonlinecitrussorting AT xiaosongan deeplearningbasedvisionsystemcombiningdetectionandtrackingforfastonlinecitrussorting AT shumingao deeplearningbasedvisionsystemcombiningdetectionandtrackingforfastonlinecitrussorting AT shanjunli deeplearningbasedvisionsystemcombiningdetectionandtrackingforfastonlinecitrussorting AT shanjunli deeplearningbasedvisionsystemcombiningdetectionandtrackingforfastonlinecitrussorting AT shanjunli deeplearningbasedvisionsystemcombiningdetectionandtrackingforfastonlinecitrussorting AT shanjunli deeplearningbasedvisionsystemcombiningdetectionandtrackingforfastonlinecitrussorting AT shanjunli deeplearningbasedvisionsystemcombiningdetectionandtrackingforfastonlinecitrussorting AT hanwenkang deeplearningbasedvisionsystemcombiningdetectionandtrackingforfastonlinecitrussorting |