Study on Multi-Heterogeneous Sensor Data Fusion Method Based on Millimeter-Wave Radar and Camera
This study presents a novel multimodal heterogeneous perception cross-fusion framework for intelligent vehicles that combines data from millimeter-wave radar and camera to enhance target tracking accuracy and handle system uncertainties. The framework employs a multimodal interaction strategy to pre...
| Main Author: | |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-06-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/23/13/6044 |
| _version_ | 1797436090327498752 |
|---|---|
| author | Jianyu Duan |
| author_facet | Jianyu Duan |
| author_sort | Jianyu Duan |
| collection | DOAJ |
| description | This study presents a novel multimodal heterogeneous perception cross-fusion framework for intelligent vehicles that combines data from millimeter-wave radar and camera to enhance target tracking accuracy and handle system uncertainties. The framework employs a multimodal interaction strategy to predict target motion more accurately and an improved joint probability data association method to match measurement data with targets. An adaptive root-mean-square cubature Kalman filter is used to estimate the statistical characteristics of noise under complex traffic scenarios with varying process and measurement noise. Experiments conducted on a real vehicle platform demonstrate that the proposed framework improves reliability and robustness in challenging environments. It overcomes the challenges of insufficient data fusion utilization, frequent leakage, and misjudgment of dangerous obstructions around vehicles, and inaccurate prediction of collision risks. The proposed framework has the potential to advance the state of the art in target tracking and perception for intelligent vehicles. |
| first_indexed | 2024-03-09T10:57:40Z |
| format | Article |
| id | doaj.art-d665dba95751485e8abe199e3e1f3dd3 |
| institution | Directory Open Access Journal |
| issn | 1424-8220 |
| language | English |
| last_indexed | 2024-03-09T10:57:40Z |
| publishDate | 2023-06-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj.art-d665dba95751485e8abe199e3e1f3dd32023-12-01T01:33:56ZengMDPI AGSensors1424-82202023-06-012313604410.3390/s23136044Study on Multi-Heterogeneous Sensor Data Fusion Method Based on Millimeter-Wave Radar and CameraJianyu Duan0School of Transportation Science and Engineering, Beihang University, Beijing 100191, ChinaThis study presents a novel multimodal heterogeneous perception cross-fusion framework for intelligent vehicles that combines data from millimeter-wave radar and camera to enhance target tracking accuracy and handle system uncertainties. The framework employs a multimodal interaction strategy to predict target motion more accurately and an improved joint probability data association method to match measurement data with targets. An adaptive root-mean-square cubature Kalman filter is used to estimate the statistical characteristics of noise under complex traffic scenarios with varying process and measurement noise. Experiments conducted on a real vehicle platform demonstrate that the proposed framework improves reliability and robustness in challenging environments. It overcomes the challenges of insufficient data fusion utilization, frequent leakage, and misjudgment of dangerous obstructions around vehicles, and inaccurate prediction of collision risks. The proposed framework has the potential to advance the state of the art in target tracking and perception for intelligent vehicles.https://www.mdpi.com/1424-8220/23/13/6044autonomous vehiclesensor fusionuncertaintyperception sensorscameraradar |
| spellingShingle | Jianyu Duan Study on Multi-Heterogeneous Sensor Data Fusion Method Based on Millimeter-Wave Radar and Camera Sensors autonomous vehicle sensor fusion uncertainty perception sensors camera radar |
| title | Study on Multi-Heterogeneous Sensor Data Fusion Method Based on Millimeter-Wave Radar and Camera |
| title_full | Study on Multi-Heterogeneous Sensor Data Fusion Method Based on Millimeter-Wave Radar and Camera |
| title_fullStr | Study on Multi-Heterogeneous Sensor Data Fusion Method Based on Millimeter-Wave Radar and Camera |
| title_full_unstemmed | Study on Multi-Heterogeneous Sensor Data Fusion Method Based on Millimeter-Wave Radar and Camera |
| title_short | Study on Multi-Heterogeneous Sensor Data Fusion Method Based on Millimeter-Wave Radar and Camera |
| title_sort | study on multi heterogeneous sensor data fusion method based on millimeter wave radar and camera |
| topic | autonomous vehicle sensor fusion uncertainty perception sensors camera radar |
| url | https://www.mdpi.com/1424-8220/23/13/6044 |
| work_keys_str_mv | AT jianyuduan studyonmultiheterogeneoussensordatafusionmethodbasedonmillimeterwaveradarandcamera |