An Intelligent Real-Time Object Detection System on Drones
Drones have been widely used in everyday life and they can help deal with various tasks, including photography, searching, and surveillance. Nonetheless, it is difficult for drones to perform customized online real-time object detection. In this study, we propose an intelligent real-time object dete...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2022-10-01
|
| Series: | Applied Sciences |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2076-3417/12/20/10227 |
| _version_ | 1797475732371275776 |
|---|---|
| author | Chao Chen Hongrui Min Yi Peng Yongkui Yang Zheng Wang |
| author_facet | Chao Chen Hongrui Min Yi Peng Yongkui Yang Zheng Wang |
| author_sort | Chao Chen |
| collection | DOAJ |
| description | Drones have been widely used in everyday life and they can help deal with various tasks, including photography, searching, and surveillance. Nonetheless, it is difficult for drones to perform customized online real-time object detection. In this study, we propose an intelligent real-time object detection system for drones. It is composed of an FPGA and a drone. A neural-network (NN) engine is designed on the FPGA for NN model acceleration. The FPGA receives activation data from an NN model, which are assembled into the data stream. Multiple fetch and jump pointers catch required activation values from the data stream, which are then filtered and sent to each thread independently. To accelerate processing speed, multiple processing elements (PEs) deal with tasks in parallel by using multiple weights and threads. The image data are transferred from the drone host to the FPGA, which are tackled with high speed by the NN engine. The NN engine results are returned to the host, which is used to adjust the flying route accordingly. Experimental results reveal that our proposed FPGA design well utilizes FPGA computing resources with 81.56% DSP and 72.80% LUT utilization rates, respectively. By using the Yolov3-tiny model for fast object detection, our system can detect objects at the speed of 8 frames per second and achieves a much lower power consumption compared to state-of-the-art methods. More importantly, the intelligent object detection techniques provide more pixels for the target of interest and they can increase the detection confidence score from 0.74 to 0.90 and from 0.70 to 0.84 for persons and cars, respectively. |
| first_indexed | 2024-03-09T20:48:06Z |
| format | Article |
| id | doaj.art-e7121936611c45d1a61a472e0789d01f |
| institution | Directory Open Access Journal |
| issn | 2076-3417 |
| language | English |
| last_indexed | 2024-03-09T20:48:06Z |
| publishDate | 2022-10-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Applied Sciences |
| spelling | doaj.art-e7121936611c45d1a61a472e0789d01f2023-11-23T22:41:10ZengMDPI AGApplied Sciences2076-34172022-10-0112201022710.3390/app122010227An Intelligent Real-Time Object Detection System on DronesChao Chen0Hongrui Min1Yi Peng2Yongkui Yang3Zheng Wang4Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen 518055, ChinaShenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen 518055, ChinaShenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen 518055, ChinaShenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen 518055, ChinaShenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen 518055, ChinaDrones have been widely used in everyday life and they can help deal with various tasks, including photography, searching, and surveillance. Nonetheless, it is difficult for drones to perform customized online real-time object detection. In this study, we propose an intelligent real-time object detection system for drones. It is composed of an FPGA and a drone. A neural-network (NN) engine is designed on the FPGA for NN model acceleration. The FPGA receives activation data from an NN model, which are assembled into the data stream. Multiple fetch and jump pointers catch required activation values from the data stream, which are then filtered and sent to each thread independently. To accelerate processing speed, multiple processing elements (PEs) deal with tasks in parallel by using multiple weights and threads. The image data are transferred from the drone host to the FPGA, which are tackled with high speed by the NN engine. The NN engine results are returned to the host, which is used to adjust the flying route accordingly. Experimental results reveal that our proposed FPGA design well utilizes FPGA computing resources with 81.56% DSP and 72.80% LUT utilization rates, respectively. By using the Yolov3-tiny model for fast object detection, our system can detect objects at the speed of 8 frames per second and achieves a much lower power consumption compared to state-of-the-art methods. More importantly, the intelligent object detection techniques provide more pixels for the target of interest and they can increase the detection confidence score from 0.74 to 0.90 and from 0.70 to 0.84 for persons and cars, respectively.https://www.mdpi.com/2076-3417/12/20/10227neural network acceleratorFPGAobject detectionintelligent systemmachine learning |
| spellingShingle | Chao Chen Hongrui Min Yi Peng Yongkui Yang Zheng Wang An Intelligent Real-Time Object Detection System on Drones Applied Sciences neural network accelerator FPGA object detection intelligent system machine learning |
| title | An Intelligent Real-Time Object Detection System on Drones |
| title_full | An Intelligent Real-Time Object Detection System on Drones |
| title_fullStr | An Intelligent Real-Time Object Detection System on Drones |
| title_full_unstemmed | An Intelligent Real-Time Object Detection System on Drones |
| title_short | An Intelligent Real-Time Object Detection System on Drones |
| title_sort | intelligent real time object detection system on drones |
| topic | neural network accelerator FPGA object detection intelligent system machine learning |
| url | https://www.mdpi.com/2076-3417/12/20/10227 |
| work_keys_str_mv | AT chaochen anintelligentrealtimeobjectdetectionsystemondrones AT hongruimin anintelligentrealtimeobjectdetectionsystemondrones AT yipeng anintelligentrealtimeobjectdetectionsystemondrones AT yongkuiyang anintelligentrealtimeobjectdetectionsystemondrones AT zhengwang anintelligentrealtimeobjectdetectionsystemondrones AT chaochen intelligentrealtimeobjectdetectionsystemondrones AT hongruimin intelligentrealtimeobjectdetectionsystemondrones AT yipeng intelligentrealtimeobjectdetectionsystemondrones AT yongkuiyang intelligentrealtimeobjectdetectionsystemondrones AT zhengwang intelligentrealtimeobjectdetectionsystemondrones |