Unifying Obstacle Detection, Recognition, and Fusion Based on the Polarization Color Stereo Camera and LiDAR for the ADAS
The perception module plays an important role in vehicles equipped with advanced driver-assistance systems (ADAS). This paper presents a multi-sensor data fusion system based on the polarization color stereo camera and the forward-looking light detection and ranging (LiDAR), which achieves the multi...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2022-03-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/22/7/2453 |
| _version_ | 1797437779313950720 |
|---|---|
| author | Ningbo Long Han Yan Liqiang Wang Haifeng Li Qing Yang |
| author_facet | Ningbo Long Han Yan Liqiang Wang Haifeng Li Qing Yang |
| author_sort | Ningbo Long |
| collection | DOAJ |
| description | The perception module plays an important role in vehicles equipped with advanced driver-assistance systems (ADAS). This paper presents a multi-sensor data fusion system based on the polarization color stereo camera and the forward-looking light detection and ranging (LiDAR), which achieves the multiple target detection, recognition, and data fusion. The You Only Look Once v4 (YOLOv4) network is utilized to achieve object detection and recognition on the color images. The depth images are obtained from the rectified left and right images based on the principle of the epipolar constraints, then the obstacles are detected from the depth images using the MeanShift algorithm. The pixel-level polarization images are extracted from the raw polarization-grey images, then the water hazards are detected successfully. The PointPillars network is employed to detect the objects from the point cloud. The calibration and synchronization between the sensors are accomplished. The experiment results show that the data fusion enriches the detection results, provides high-dimensional perceptual information and extends the effective detection range. Meanwhile, the detection results are stable under diverse range and illumination conditions. |
| first_indexed | 2024-03-09T11:27:32Z |
| format | Article |
| id | doaj.art-692c3e1937d047cfb90ac02c1b4bcd96 |
| institution | Directory Open Access Journal |
| issn | 1424-8220 |
| language | English |
| last_indexed | 2024-03-09T11:27:32Z |
| publishDate | 2022-03-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj.art-692c3e1937d047cfb90ac02c1b4bcd962023-11-30T23:59:06ZengMDPI AGSensors1424-82202022-03-01227245310.3390/s22072453Unifying Obstacle Detection, Recognition, and Fusion Based on the Polarization Color Stereo Camera and LiDAR for the ADASNingbo Long0Han Yan1Liqiang Wang2Haifeng Li3Qing Yang4Research Center for Humanoid Sensing, Zhejiang Lab, Hangzhou 311100, ChinaScience and Technology on Space Intelligent Control Laboratory, Beijing Institute of Control Engineering, Beijing 100094, ChinaResearch Center for Humanoid Sensing, Zhejiang Lab, Hangzhou 311100, ChinaResearch Center for Humanoid Sensing, Zhejiang Lab, Hangzhou 311100, ChinaResearch Center for Humanoid Sensing, Zhejiang Lab, Hangzhou 311100, ChinaThe perception module plays an important role in vehicles equipped with advanced driver-assistance systems (ADAS). This paper presents a multi-sensor data fusion system based on the polarization color stereo camera and the forward-looking light detection and ranging (LiDAR), which achieves the multiple target detection, recognition, and data fusion. The You Only Look Once v4 (YOLOv4) network is utilized to achieve object detection and recognition on the color images. The depth images are obtained from the rectified left and right images based on the principle of the epipolar constraints, then the obstacles are detected from the depth images using the MeanShift algorithm. The pixel-level polarization images are extracted from the raw polarization-grey images, then the water hazards are detected successfully. The PointPillars network is employed to detect the objects from the point cloud. The calibration and synchronization between the sensors are accomplished. The experiment results show that the data fusion enriches the detection results, provides high-dimensional perceptual information and extends the effective detection range. Meanwhile, the detection results are stable under diverse range and illumination conditions.https://www.mdpi.com/1424-8220/22/7/2453polarization-color-depthstereo cameranon-repetitive scanning LiDARsensor fusion |
| spellingShingle | Ningbo Long Han Yan Liqiang Wang Haifeng Li Qing Yang Unifying Obstacle Detection, Recognition, and Fusion Based on the Polarization Color Stereo Camera and LiDAR for the ADAS Sensors polarization-color-depth stereo camera non-repetitive scanning LiDAR sensor fusion |
| title | Unifying Obstacle Detection, Recognition, and Fusion Based on the Polarization Color Stereo Camera and LiDAR for the ADAS |
| title_full | Unifying Obstacle Detection, Recognition, and Fusion Based on the Polarization Color Stereo Camera and LiDAR for the ADAS |
| title_fullStr | Unifying Obstacle Detection, Recognition, and Fusion Based on the Polarization Color Stereo Camera and LiDAR for the ADAS |
| title_full_unstemmed | Unifying Obstacle Detection, Recognition, and Fusion Based on the Polarization Color Stereo Camera and LiDAR for the ADAS |
| title_short | Unifying Obstacle Detection, Recognition, and Fusion Based on the Polarization Color Stereo Camera and LiDAR for the ADAS |
| title_sort | unifying obstacle detection recognition and fusion based on the polarization color stereo camera and lidar for the adas |
| topic | polarization-color-depth stereo camera non-repetitive scanning LiDAR sensor fusion |
| url | https://www.mdpi.com/1424-8220/22/7/2453 |
| work_keys_str_mv | AT ningbolong unifyingobstacledetectionrecognitionandfusionbasedonthepolarizationcolorstereocameraandlidarfortheadas AT hanyan unifyingobstacledetectionrecognitionandfusionbasedonthepolarizationcolorstereocameraandlidarfortheadas AT liqiangwang unifyingobstacledetectionrecognitionandfusionbasedonthepolarizationcolorstereocameraandlidarfortheadas AT haifengli unifyingobstacledetectionrecognitionandfusionbasedonthepolarizationcolorstereocameraandlidarfortheadas AT qingyang unifyingobstacledetectionrecognitionandfusionbasedonthepolarizationcolorstereocameraandlidarfortheadas |