Low-Power Dynamic Object Detection and Classification With Freely Moving Event Cameras
We present the first purely event-based, energy-efficient approach for dynamic object detection and categorization with a freely moving event camera. Compared to traditional cameras, event-based object recognition systems are considerably behind in terms of accuracy and algorithmic maturity. To this...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Frontiers Media S.A.
2020-02-01
|
| Series: | Frontiers in Neuroscience |
| Subjects: | |
| Online Access: | https://www.frontiersin.org/article/10.3389/fnins.2020.00135/full |
| _version_ | 1819051480666079232 |
|---|---|
| author | Bharath Ramesh Bharath Ramesh Andrés Ussa Andrés Ussa Luca Della Vedova Hong Yang Garrick Orchard Garrick Orchard |
| author_facet | Bharath Ramesh Bharath Ramesh Andrés Ussa Andrés Ussa Luca Della Vedova Hong Yang Garrick Orchard Garrick Orchard |
| author_sort | Bharath Ramesh |
| collection | DOAJ |
| description | We present the first purely event-based, energy-efficient approach for dynamic object detection and categorization with a freely moving event camera. Compared to traditional cameras, event-based object recognition systems are considerably behind in terms of accuracy and algorithmic maturity. To this end, this paper presents an event-based feature extraction method devised by accumulating local activity across the image frame and then applying principal component analysis (PCA) to the normalized neighborhood region. Subsequently, we propose a backtracking-free k-d tree mechanism for efficient feature matching by taking advantage of the low-dimensionality of the feature representation. Additionally, the proposed k-d tree mechanism allows for feature selection to obtain a lower-dimensional object representation when hardware resources are limited to implement PCA. Consequently, the proposed system can be realized on a field-programmable gate array (FPGA) device leading to high performance over resource ratio. The proposed system is tested on real-world event-based datasets for object categorization, showing superior classification performance compared to state-of-the-art algorithms. Additionally, we verified the real-time FPGA performance of the proposed object detection method, trained with limited data as opposed to deep learning methods, under a closed-loop aerial vehicle flight mode. We also compare the proposed object categorization framework to pre-trained convolutional neural networks using transfer learning and highlight the drawbacks of using frame-based sensors under dynamic camera motion. Finally, we provide critical insights about the feature extraction method and the classification parameters on the system performance, which aids in understanding the framework to suit various low-power (less than a few watts) application scenarios. |
| first_indexed | 2024-12-21T12:04:37Z |
| format | Article |
| id | doaj.art-451c81afa2b843d2ac67a09570078a0c |
| institution | Directory Open Access Journal |
| issn | 1662-453X |
| language | English |
| last_indexed | 2024-12-21T12:04:37Z |
| publishDate | 2020-02-01 |
| publisher | Frontiers Media S.A. |
| record_format | Article |
| series | Frontiers in Neuroscience |
| spelling | doaj.art-451c81afa2b843d2ac67a09570078a0c2022-12-21T19:04:43ZengFrontiers Media S.A.Frontiers in Neuroscience1662-453X2020-02-011410.3389/fnins.2020.00135505328Low-Power Dynamic Object Detection and Classification With Freely Moving Event CamerasBharath Ramesh0Bharath Ramesh1Andrés Ussa2Andrés Ussa3Luca Della Vedova4Hong Yang5Garrick Orchard6Garrick Orchard7Life Science Institute, The N.1 Institute for Health, National University of Singapore, Singapore, SingaporeTemasek Laboratories, National University of Singapore, Singapore, SingaporeLife Science Institute, The N.1 Institute for Health, National University of Singapore, Singapore, SingaporeTemasek Laboratories, National University of Singapore, Singapore, SingaporeTemasek Laboratories, National University of Singapore, Singapore, SingaporeTemasek Laboratories, National University of Singapore, Singapore, SingaporeLife Science Institute, The N.1 Institute for Health, National University of Singapore, Singapore, SingaporeTemasek Laboratories, National University of Singapore, Singapore, SingaporeWe present the first purely event-based, energy-efficient approach for dynamic object detection and categorization with a freely moving event camera. Compared to traditional cameras, event-based object recognition systems are considerably behind in terms of accuracy and algorithmic maturity. To this end, this paper presents an event-based feature extraction method devised by accumulating local activity across the image frame and then applying principal component analysis (PCA) to the normalized neighborhood region. Subsequently, we propose a backtracking-free k-d tree mechanism for efficient feature matching by taking advantage of the low-dimensionality of the feature representation. Additionally, the proposed k-d tree mechanism allows for feature selection to obtain a lower-dimensional object representation when hardware resources are limited to implement PCA. Consequently, the proposed system can be realized on a field-programmable gate array (FPGA) device leading to high performance over resource ratio. The proposed system is tested on real-world event-based datasets for object categorization, showing superior classification performance compared to state-of-the-art algorithms. Additionally, we verified the real-time FPGA performance of the proposed object detection method, trained with limited data as opposed to deep learning methods, under a closed-loop aerial vehicle flight mode. We also compare the proposed object categorization framework to pre-trained convolutional neural networks using transfer learning and highlight the drawbacks of using frame-based sensors under dynamic camera motion. Finally, we provide critical insights about the feature extraction method and the classification parameters on the system performance, which aids in understanding the framework to suit various low-power (less than a few watts) application scenarios.https://www.frontiersin.org/article/10.3389/fnins.2020.00135/fullobject recognitionneuromorphic visionlow-power FPGAclosed-loop controlobject detectionevent-based descriptor |
| spellingShingle | Bharath Ramesh Bharath Ramesh Andrés Ussa Andrés Ussa Luca Della Vedova Hong Yang Garrick Orchard Garrick Orchard Low-Power Dynamic Object Detection and Classification With Freely Moving Event Cameras Frontiers in Neuroscience object recognition neuromorphic vision low-power FPGA closed-loop control object detection event-based descriptor |
| title | Low-Power Dynamic Object Detection and Classification With Freely Moving Event Cameras |
| title_full | Low-Power Dynamic Object Detection and Classification With Freely Moving Event Cameras |
| title_fullStr | Low-Power Dynamic Object Detection and Classification With Freely Moving Event Cameras |
| title_full_unstemmed | Low-Power Dynamic Object Detection and Classification With Freely Moving Event Cameras |
| title_short | Low-Power Dynamic Object Detection and Classification With Freely Moving Event Cameras |
| title_sort | low power dynamic object detection and classification with freely moving event cameras |
| topic | object recognition neuromorphic vision low-power FPGA closed-loop control object detection event-based descriptor |
| url | https://www.frontiersin.org/article/10.3389/fnins.2020.00135/full |
| work_keys_str_mv | AT bharathramesh lowpowerdynamicobjectdetectionandclassificationwithfreelymovingeventcameras AT bharathramesh lowpowerdynamicobjectdetectionandclassificationwithfreelymovingeventcameras AT andresussa lowpowerdynamicobjectdetectionandclassificationwithfreelymovingeventcameras AT andresussa lowpowerdynamicobjectdetectionandclassificationwithfreelymovingeventcameras AT lucadellavedova lowpowerdynamicobjectdetectionandclassificationwithfreelymovingeventcameras AT hongyang lowpowerdynamicobjectdetectionandclassificationwithfreelymovingeventcameras AT garrickorchard lowpowerdynamicobjectdetectionandclassificationwithfreelymovingeventcameras AT garrickorchard lowpowerdynamicobjectdetectionandclassificationwithfreelymovingeventcameras |