A Deep Learning-Based Vision System Combining Detection and Tracking for Fast On-Line Citrus Sorting

Defective citrus fruits are manually sorted at the moment, which is a time-consuming and cost-expensive process with unsatisfactory accuracy. In this paper, we introduce a deep learning-based vision system implemented on a citrus processing line for fast on-line sorting. For the citrus fruits rotati...

Full description

Bibliographic Details
Main Authors: Yaohui Chen, Xiaosong An, Shumin Gao, Shanjun Li, Hanwen Kang
Format: Article
Language:English
Published: Frontiers Media S.A. 2021-02-01
Series:Frontiers in Plant Science
Subjects:
Online Access:https://www.frontiersin.org/articles/10.3389/fpls.2021.622062/full
_version_ 1818677344027541504
author Yaohui Chen
Yaohui Chen
Yaohui Chen
Xiaosong An
Shumin Gao
Shanjun Li
Shanjun Li
Shanjun Li
Shanjun Li
Shanjun Li
Hanwen Kang
author_facet Yaohui Chen
Yaohui Chen
Yaohui Chen
Xiaosong An
Shumin Gao
Shanjun Li
Shanjun Li
Shanjun Li
Shanjun Li
Shanjun Li
Hanwen Kang
author_sort Yaohui Chen
collection DOAJ
description Defective citrus fruits are manually sorted at the moment, which is a time-consuming and cost-expensive process with unsatisfactory accuracy. In this paper, we introduce a deep learning-based vision system implemented on a citrus processing line for fast on-line sorting. For the citrus fruits rotating randomly on the conveyor, a convolutional neural network-based detector was developed to detect and temporarily classify the defective ones, and a SORT algorithm-based tracker was adopted to record the classification information along their paths. The true categories of the citrus fruits were identified through the tracked historical information, resulting in high detection precision of 93.6%. Moreover, the linear Kalman filter model was applied to predict the future path of the fruits, which can be used to guide the robot arms to pick out the defective ones. Ultimately, this research presents a practical solution to realize on-line citrus sorting featuring low costs, high efficiency, and accuracy.
first_indexed 2024-12-17T08:57:52Z
format Article
id doaj.art-5de64c770df24ff19c32185a5815d484
institution Directory Open Access Journal
issn 1664-462X
language English
last_indexed 2024-12-17T08:57:52Z
publishDate 2021-02-01
publisher Frontiers Media S.A.
record_format Article
series Frontiers in Plant Science
spelling doaj.art-5de64c770df24ff19c32185a5815d4842022-12-21T21:55:52ZengFrontiers Media S.A.Frontiers in Plant Science1664-462X2021-02-011210.3389/fpls.2021.622062622062A Deep Learning-Based Vision System Combining Detection and Tracking for Fast On-Line Citrus SortingYaohui Chen0Yaohui Chen1Yaohui Chen2Xiaosong An3Shumin Gao4Shanjun Li5Shanjun Li6Shanjun Li7Shanjun Li8Shanjun Li9Hanwen Kang10College of Engineering, Huazhong Agricultural University, Wuhan, ChinaKey Laboratory of Agricultural Equipment in Mid-Lower Yangtze River, Ministry of Agriculture and Rural Affairs, Wuhan, ChinaCitrus Mechanization Research Base, Ministry of Agriculture and Rural Affairs, Wuhan, ChinaCollege of Engineering, Huazhong Agricultural University, Wuhan, ChinaCollege of Engineering, Huazhong Agricultural University, Wuhan, ChinaCollege of Engineering, Huazhong Agricultural University, Wuhan, ChinaKey Laboratory of Agricultural Equipment in Mid-Lower Yangtze River, Ministry of Agriculture and Rural Affairs, Wuhan, ChinaCitrus Mechanization Research Base, Ministry of Agriculture and Rural Affairs, Wuhan, ChinaChina Agriculture (Citrus) Research System, Wuhan, ChinaNational R&D Center for Citrus Preservation, Wuhan, ChinaDepartment of Mechanical and Aerospace Engineering, College of Engineering, Monash University, Clayton, VIC, AustraliaDefective citrus fruits are manually sorted at the moment, which is a time-consuming and cost-expensive process with unsatisfactory accuracy. In this paper, we introduce a deep learning-based vision system implemented on a citrus processing line for fast on-line sorting. For the citrus fruits rotating randomly on the conveyor, a convolutional neural network-based detector was developed to detect and temporarily classify the defective ones, and a SORT algorithm-based tracker was adopted to record the classification information along their paths. The true categories of the citrus fruits were identified through the tracked historical information, resulting in high detection precision of 93.6%. Moreover, the linear Kalman filter model was applied to predict the future path of the fruits, which can be used to guide the robot arms to pick out the defective ones. Ultimately, this research presents a practical solution to realize on-line citrus sorting featuring low costs, high efficiency, and accuracy.https://www.frontiersin.org/articles/10.3389/fpls.2021.622062/fulldefective citrus sortingCNN-based detectorSORT-based trackerdeep learningvision system
spellingShingle Yaohui Chen
Yaohui Chen
Yaohui Chen
Xiaosong An
Shumin Gao
Shanjun Li
Shanjun Li
Shanjun Li
Shanjun Li
Shanjun Li
Hanwen Kang
A Deep Learning-Based Vision System Combining Detection and Tracking for Fast On-Line Citrus Sorting
Frontiers in Plant Science
defective citrus sorting
CNN-based detector
SORT-based tracker
deep learning
vision system
title A Deep Learning-Based Vision System Combining Detection and Tracking for Fast On-Line Citrus Sorting
title_full A Deep Learning-Based Vision System Combining Detection and Tracking for Fast On-Line Citrus Sorting
title_fullStr A Deep Learning-Based Vision System Combining Detection and Tracking for Fast On-Line Citrus Sorting
title_full_unstemmed A Deep Learning-Based Vision System Combining Detection and Tracking for Fast On-Line Citrus Sorting
title_short A Deep Learning-Based Vision System Combining Detection and Tracking for Fast On-Line Citrus Sorting
title_sort deep learning based vision system combining detection and tracking for fast on line citrus sorting
topic defective citrus sorting
CNN-based detector
SORT-based tracker
deep learning
vision system
url https://www.frontiersin.org/articles/10.3389/fpls.2021.622062/full
work_keys_str_mv AT yaohuichen adeeplearningbasedvisionsystemcombiningdetectionandtrackingforfastonlinecitrussorting
AT yaohuichen adeeplearningbasedvisionsystemcombiningdetectionandtrackingforfastonlinecitrussorting
AT yaohuichen adeeplearningbasedvisionsystemcombiningdetectionandtrackingforfastonlinecitrussorting
AT xiaosongan adeeplearningbasedvisionsystemcombiningdetectionandtrackingforfastonlinecitrussorting
AT shumingao adeeplearningbasedvisionsystemcombiningdetectionandtrackingforfastonlinecitrussorting
AT shanjunli adeeplearningbasedvisionsystemcombiningdetectionandtrackingforfastonlinecitrussorting
AT shanjunli adeeplearningbasedvisionsystemcombiningdetectionandtrackingforfastonlinecitrussorting
AT shanjunli adeeplearningbasedvisionsystemcombiningdetectionandtrackingforfastonlinecitrussorting
AT shanjunli adeeplearningbasedvisionsystemcombiningdetectionandtrackingforfastonlinecitrussorting
AT shanjunli adeeplearningbasedvisionsystemcombiningdetectionandtrackingforfastonlinecitrussorting
AT hanwenkang adeeplearningbasedvisionsystemcombiningdetectionandtrackingforfastonlinecitrussorting
AT yaohuichen deeplearningbasedvisionsystemcombiningdetectionandtrackingforfastonlinecitrussorting
AT yaohuichen deeplearningbasedvisionsystemcombiningdetectionandtrackingforfastonlinecitrussorting
AT yaohuichen deeplearningbasedvisionsystemcombiningdetectionandtrackingforfastonlinecitrussorting
AT xiaosongan deeplearningbasedvisionsystemcombiningdetectionandtrackingforfastonlinecitrussorting
AT shumingao deeplearningbasedvisionsystemcombiningdetectionandtrackingforfastonlinecitrussorting
AT shanjunli deeplearningbasedvisionsystemcombiningdetectionandtrackingforfastonlinecitrussorting
AT shanjunli deeplearningbasedvisionsystemcombiningdetectionandtrackingforfastonlinecitrussorting
AT shanjunli deeplearningbasedvisionsystemcombiningdetectionandtrackingforfastonlinecitrussorting
AT shanjunli deeplearningbasedvisionsystemcombiningdetectionandtrackingforfastonlinecitrussorting
AT shanjunli deeplearningbasedvisionsystemcombiningdetectionandtrackingforfastonlinecitrussorting
AT hanwenkang deeplearningbasedvisionsystemcombiningdetectionandtrackingforfastonlinecitrussorting