FPGA-Based Feature Extraction and Tracking Accelerator for Real-Time Visual SLAM
Due to its advantages of low latency, low power consumption, and high flexibility, FPGA-based acceleration technology has been more and more widely studied and applied in the field of computer vision in recent years. An FPGA-based feature extraction and tracking accelerator for real-time visual odom...
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-09-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/23/19/8035 |
| _version_ | 1827722239555928064 |
|---|---|
| author | Jie Zhang Shuai Xiong Cheng Liu Yongchao Geng Wei Xiong Song Cheng Fang Hu |
| author_facet | Jie Zhang Shuai Xiong Cheng Liu Yongchao Geng Wei Xiong Song Cheng Fang Hu |
| author_sort | Jie Zhang |
| collection | DOAJ |
| description | Due to its advantages of low latency, low power consumption, and high flexibility, FPGA-based acceleration technology has been more and more widely studied and applied in the field of computer vision in recent years. An FPGA-based feature extraction and tracking accelerator for real-time visual odometry (VO) and visual simultaneous localization and mapping (V-SLAM) is proposed, which can realize the complete acceleration processing capability of the image front-end. For the first time, we implement a hardware solution that combines features from accelerated segment test (FAST) feature points with Gunnar Farneback (GF) dense optical flow to achieve better feature tracking performance and provide more flexible technical route selection. In order to solve the scale invariance and rotation invariance lacking problems of FAST features, an efficient pyramid module with a five-layer thumbnail structure was designed and implemented. The accelerator was implemented on a modern Xilinx Zynq FPGA. The evaluation results showed that the accelerator could achieve stable tracking of features of violently shaking images and were consistent with the results from MATLAB code running on PCs. Compared to PC CPUs, which require seconds of processing time, the processing latency was greatly reduced to the order of milliseconds, making GF dense optical flow an efficient and practical technical solution on the edge side. |
| first_indexed | 2024-03-10T21:36:19Z |
| format | Article |
| id | doaj.art-f16367848e854217a0b6323bc6d020ad |
| institution | Directory Open Access Journal |
| issn | 1424-8220 |
| language | English |
| last_indexed | 2024-03-10T21:36:19Z |
| publishDate | 2023-09-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj.art-f16367848e854217a0b6323bc6d020ad2023-11-19T15:01:46ZengMDPI AGSensors1424-82202023-09-012319803510.3390/s23198035FPGA-Based Feature Extraction and Tracking Accelerator for Real-Time Visual SLAMJie Zhang0Shuai Xiong1Cheng Liu2Yongchao Geng3Wei Xiong4Song Cheng5Fang Hu6National Astronomical Observatories, Chinese Academy of Sciences, Beijing 100101, ChinaThe 20th Research Institute of China Electronics Technology Group Corporation, Xi’an 710068, ChinaBeijing Eyestar Technology Co., Ltd., Beijing 102200, ChinaThe 20th Research Institute of China Electronics Technology Group Corporation, Xi’an 710068, ChinaBeijing Eyestar Technology Co., Ltd., Beijing 102200, ChinaThe 20th Research Institute of China Electronics Technology Group Corporation, Xi’an 710068, ChinaThe 20th Research Institute of China Electronics Technology Group Corporation, Xi’an 710068, ChinaDue to its advantages of low latency, low power consumption, and high flexibility, FPGA-based acceleration technology has been more and more widely studied and applied in the field of computer vision in recent years. An FPGA-based feature extraction and tracking accelerator for real-time visual odometry (VO) and visual simultaneous localization and mapping (V-SLAM) is proposed, which can realize the complete acceleration processing capability of the image front-end. For the first time, we implement a hardware solution that combines features from accelerated segment test (FAST) feature points with Gunnar Farneback (GF) dense optical flow to achieve better feature tracking performance and provide more flexible technical route selection. In order to solve the scale invariance and rotation invariance lacking problems of FAST features, an efficient pyramid module with a five-layer thumbnail structure was designed and implemented. The accelerator was implemented on a modern Xilinx Zynq FPGA. The evaluation results showed that the accelerator could achieve stable tracking of features of violently shaking images and were consistent with the results from MATLAB code running on PCs. Compared to PC CPUs, which require seconds of processing time, the processing latency was greatly reduced to the order of milliseconds, making GF dense optical flow an efficient and practical technical solution on the edge side.https://www.mdpi.com/1424-8220/23/19/8035VIOV-SLAMFPGAhistogram equalizationFASTpyramid processing |
| spellingShingle | Jie Zhang Shuai Xiong Cheng Liu Yongchao Geng Wei Xiong Song Cheng Fang Hu FPGA-Based Feature Extraction and Tracking Accelerator for Real-Time Visual SLAM Sensors VIO V-SLAM FPGA histogram equalization FAST pyramid processing |
| title | FPGA-Based Feature Extraction and Tracking Accelerator for Real-Time Visual SLAM |
| title_full | FPGA-Based Feature Extraction and Tracking Accelerator for Real-Time Visual SLAM |
| title_fullStr | FPGA-Based Feature Extraction and Tracking Accelerator for Real-Time Visual SLAM |
| title_full_unstemmed | FPGA-Based Feature Extraction and Tracking Accelerator for Real-Time Visual SLAM |
| title_short | FPGA-Based Feature Extraction and Tracking Accelerator for Real-Time Visual SLAM |
| title_sort | fpga based feature extraction and tracking accelerator for real time visual slam |
| topic | VIO V-SLAM FPGA histogram equalization FAST pyramid processing |
| url | https://www.mdpi.com/1424-8220/23/19/8035 |
| work_keys_str_mv | AT jiezhang fpgabasedfeatureextractionandtrackingacceleratorforrealtimevisualslam AT shuaixiong fpgabasedfeatureextractionandtrackingacceleratorforrealtimevisualslam AT chengliu fpgabasedfeatureextractionandtrackingacceleratorforrealtimevisualslam AT yongchaogeng fpgabasedfeatureextractionandtrackingacceleratorforrealtimevisualslam AT weixiong fpgabasedfeatureextractionandtrackingacceleratorforrealtimevisualslam AT songcheng fpgabasedfeatureextractionandtrackingacceleratorforrealtimevisualslam AT fanghu fpgabasedfeatureextractionandtrackingacceleratorforrealtimevisualslam |