Enhancing Embedded Object Tracking: A Hardware Acceleration Approach for Real-Time Predictability
While Siamese object tracking has witnessed significant advancements, its hard real-time behaviour on embedded devices remains inadequately addressed. In many application cases, an embedded implementation should not only have a minimal execution latency, but this latency should ideally also have zer...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2024-03-01
|
| Series: | Journal of Imaging |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2313-433X/10/3/70 |
| _version_ | 1797240484667588608 |
|---|---|
| author | Mingyang Zhang Kristof Van Beeck Toon Goedemé |
| author_facet | Mingyang Zhang Kristof Van Beeck Toon Goedemé |
| author_sort | Mingyang Zhang |
| collection | DOAJ |
| description | While Siamese object tracking has witnessed significant advancements, its hard real-time behaviour on embedded devices remains inadequately addressed. In many application cases, an embedded implementation should not only have a minimal execution latency, but this latency should ideally also have zero variance, i.e., be predictable. This study aims to address this issue by meticulously analysing real-time predictability across different components of a deep-learning-based video object tracking system. Our detailed experiments not only indicate the superiority of Field-Programmable Gate Array (FPGA) implementations in terms of hard real-time behaviour but also unveil important time predictability bottlenecks. We introduce dedicated hardware accelerators for key processes, focusing on depth-wise cross-correlation and padding operations, utilizing high-level synthesis (HLS). Implemented on a KV260 board, our enhanced tracker exhibits not only a speed up, with a factor of 6.6, in mean execution time but also significant improvements in hard real-time predictability by yielding 11 times less latency variation as compared to our baseline. A subsequent analysis of power consumption reveals our approach’s contribution to enhanced power efficiency. These advancements underscore the crucial role of hardware acceleration in realizing time-predictable object tracking on embedded systems, setting new standards for future hardware–software co-design endeavours in this domain. |
| first_indexed | 2024-04-24T18:08:10Z |
| format | Article |
| id | doaj.art-7e0ad21380d649c8832bc006263d6cab |
| institution | Directory Open Access Journal |
| issn | 2313-433X |
| language | English |
| last_indexed | 2024-04-24T18:08:10Z |
| publishDate | 2024-03-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Journal of Imaging |
| spelling | doaj.art-7e0ad21380d649c8832bc006263d6cab2024-03-27T13:48:53ZengMDPI AGJournal of Imaging2313-433X2024-03-011037010.3390/jimaging10030070Enhancing Embedded Object Tracking: A Hardware Acceleration Approach for Real-Time PredictabilityMingyang Zhang0Kristof Van Beeck1Toon Goedemé2PSI-EAVISE Research Group, Department of Electrical Engineering, KU Leuven, 2860 Sint-Katelijne-Waver, BelgiumPSI-EAVISE Research Group, Department of Electrical Engineering, KU Leuven, 2860 Sint-Katelijne-Waver, BelgiumPSI-EAVISE Research Group, Department of Electrical Engineering, KU Leuven, 2860 Sint-Katelijne-Waver, BelgiumWhile Siamese object tracking has witnessed significant advancements, its hard real-time behaviour on embedded devices remains inadequately addressed. In many application cases, an embedded implementation should not only have a minimal execution latency, but this latency should ideally also have zero variance, i.e., be predictable. This study aims to address this issue by meticulously analysing real-time predictability across different components of a deep-learning-based video object tracking system. Our detailed experiments not only indicate the superiority of Field-Programmable Gate Array (FPGA) implementations in terms of hard real-time behaviour but also unveil important time predictability bottlenecks. We introduce dedicated hardware accelerators for key processes, focusing on depth-wise cross-correlation and padding operations, utilizing high-level synthesis (HLS). Implemented on a KV260 board, our enhanced tracker exhibits not only a speed up, with a factor of 6.6, in mean execution time but also significant improvements in hard real-time predictability by yielding 11 times less latency variation as compared to our baseline. A subsequent analysis of power consumption reveals our approach’s contribution to enhanced power efficiency. These advancements underscore the crucial role of hardware acceleration in realizing time-predictable object tracking on embedded systems, setting new standards for future hardware–software co-design endeavours in this domain.https://www.mdpi.com/2313-433X/10/3/70deep learningobject trackingsiamese networkFPGAreal-time system predictabilityhardware acceleration |
| spellingShingle | Mingyang Zhang Kristof Van Beeck Toon Goedemé Enhancing Embedded Object Tracking: A Hardware Acceleration Approach for Real-Time Predictability Journal of Imaging deep learning object tracking siamese network FPGA real-time system predictability hardware acceleration |
| title | Enhancing Embedded Object Tracking: A Hardware Acceleration Approach for Real-Time Predictability |
| title_full | Enhancing Embedded Object Tracking: A Hardware Acceleration Approach for Real-Time Predictability |
| title_fullStr | Enhancing Embedded Object Tracking: A Hardware Acceleration Approach for Real-Time Predictability |
| title_full_unstemmed | Enhancing Embedded Object Tracking: A Hardware Acceleration Approach for Real-Time Predictability |
| title_short | Enhancing Embedded Object Tracking: A Hardware Acceleration Approach for Real-Time Predictability |
| title_sort | enhancing embedded object tracking a hardware acceleration approach for real time predictability |
| topic | deep learning object tracking siamese network FPGA real-time system predictability hardware acceleration |
| url | https://www.mdpi.com/2313-433X/10/3/70 |
| work_keys_str_mv | AT mingyangzhang enhancingembeddedobjecttrackingahardwareaccelerationapproachforrealtimepredictability AT kristofvanbeeck enhancingembeddedobjecttrackingahardwareaccelerationapproachforrealtimepredictability AT toongoedeme enhancingembeddedobjecttrackingahardwareaccelerationapproachforrealtimepredictability |