MME-YOLO: Multi-Sensor Multi-Level Enhanced YOLO for Robust Vehicle Detection in Traffic Surveillance
As an effective means of solving collision problems caused by the limited perspective on board, the cooperative roadside system is gaining popularity. To improve the vehicle detection abilities in such online safety systems, in this paper, we propose a novel multi-sensor multi-level enhanced convolu...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2020-12-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/21/1/27 |
| _version_ | 1797543788684509184 |
|---|---|
| author | Jianxiao Zhu Xu Li Peng Jin Qimin Xu Zhengliang Sun Xiang Song |
| author_facet | Jianxiao Zhu Xu Li Peng Jin Qimin Xu Zhengliang Sun Xiang Song |
| author_sort | Jianxiao Zhu |
| collection | DOAJ |
| description | As an effective means of solving collision problems caused by the limited perspective on board, the cooperative roadside system is gaining popularity. To improve the vehicle detection abilities in such online safety systems, in this paper, we propose a novel multi-sensor multi-level enhanced convolutional network model, called multi-sensor multi-level enhanced convolutional network architecture (MME-YOLO), with consideration of hybrid realistic scene of scales, illumination, and occlusion. MME-YOLO consists of two tightly coupled structures, i.e., the enhanced inference head and the LiDAR-Image composite module. More specifically, the enhanced inference head preliminarily equips the network with stronger inference abilities for redundant visual cues by attention-guided feature selection blocks and anchor-based/anchor-free ensemble head. Furthermore, the LiDAR-Image composite module cascades the multi-level feature maps from the LiDAR subnet to the image subnet, which strengthens the generalization of the detector in complex scenarios. Compared with YOLOv3, the enhanced inference head achieves a 5.83% and 4.88% mAP improvement on visual dataset LVSH and UA-DETRAC, respectively. Integrated with the composite module, the overall architecture gains 91.63% mAP in the collected Road-side Dataset. Experiments show that even under the abnormal lightings and the inconsistent scales at evening rush hours, the proposed MME-YOLO maintains reliable recognition accuracy and robust detection performance. |
| first_indexed | 2024-03-10T13:50:34Z |
| format | Article |
| id | doaj.art-a47a8aa1498348c5a68f460a3fc7a4d8 |
| institution | Directory Open Access Journal |
| issn | 1424-8220 |
| language | English |
| last_indexed | 2024-03-10T13:50:34Z |
| publishDate | 2020-12-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj.art-a47a8aa1498348c5a68f460a3fc7a4d82023-11-21T02:11:05ZengMDPI AGSensors1424-82202020-12-012112710.3390/s21010027MME-YOLO: Multi-Sensor Multi-Level Enhanced YOLO for Robust Vehicle Detection in Traffic SurveillanceJianxiao Zhu0Xu Li1Peng Jin2Qimin Xu3Zhengliang Sun4Xiang Song5School of Instrument Science and Engineering, Southeast University, Nanjing 210096, ChinaSchool of Instrument Science and Engineering, Southeast University, Nanjing 210096, ChinaSchool of Instrument Science and Engineering, Southeast University, Nanjing 210096, ChinaSchool of Instrument Science and Engineering, Southeast University, Nanjing 210096, ChinaTraffic Management Research Institute, Ministry of Public Security, Wuxi 214151, ChinaSchool of Electronic Engineering, Nanjing Xiaozhuang University, Nanjing 211171, ChinaAs an effective means of solving collision problems caused by the limited perspective on board, the cooperative roadside system is gaining popularity. To improve the vehicle detection abilities in such online safety systems, in this paper, we propose a novel multi-sensor multi-level enhanced convolutional network model, called multi-sensor multi-level enhanced convolutional network architecture (MME-YOLO), with consideration of hybrid realistic scene of scales, illumination, and occlusion. MME-YOLO consists of two tightly coupled structures, i.e., the enhanced inference head and the LiDAR-Image composite module. More specifically, the enhanced inference head preliminarily equips the network with stronger inference abilities for redundant visual cues by attention-guided feature selection blocks and anchor-based/anchor-free ensemble head. Furthermore, the LiDAR-Image composite module cascades the multi-level feature maps from the LiDAR subnet to the image subnet, which strengthens the generalization of the detector in complex scenarios. Compared with YOLOv3, the enhanced inference head achieves a 5.83% and 4.88% mAP improvement on visual dataset LVSH and UA-DETRAC, respectively. Integrated with the composite module, the overall architecture gains 91.63% mAP in the collected Road-side Dataset. Experiments show that even under the abnormal lightings and the inconsistent scales at evening rush hours, the proposed MME-YOLO maintains reliable recognition accuracy and robust detection performance.https://www.mdpi.com/1424-8220/21/1/27vehicle detectionmulti-sensor fusioncomplex scenesmulti-scalessmart city |
| spellingShingle | Jianxiao Zhu Xu Li Peng Jin Qimin Xu Zhengliang Sun Xiang Song MME-YOLO: Multi-Sensor Multi-Level Enhanced YOLO for Robust Vehicle Detection in Traffic Surveillance Sensors vehicle detection multi-sensor fusion complex scenes multi-scales smart city |
| title | MME-YOLO: Multi-Sensor Multi-Level Enhanced YOLO for Robust Vehicle Detection in Traffic Surveillance |
| title_full | MME-YOLO: Multi-Sensor Multi-Level Enhanced YOLO for Robust Vehicle Detection in Traffic Surveillance |
| title_fullStr | MME-YOLO: Multi-Sensor Multi-Level Enhanced YOLO for Robust Vehicle Detection in Traffic Surveillance |
| title_full_unstemmed | MME-YOLO: Multi-Sensor Multi-Level Enhanced YOLO for Robust Vehicle Detection in Traffic Surveillance |
| title_short | MME-YOLO: Multi-Sensor Multi-Level Enhanced YOLO for Robust Vehicle Detection in Traffic Surveillance |
| title_sort | mme yolo multi sensor multi level enhanced yolo for robust vehicle detection in traffic surveillance |
| topic | vehicle detection multi-sensor fusion complex scenes multi-scales smart city |
| url | https://www.mdpi.com/1424-8220/21/1/27 |
| work_keys_str_mv | AT jianxiaozhu mmeyolomultisensormultilevelenhancedyoloforrobustvehicledetectionintrafficsurveillance AT xuli mmeyolomultisensormultilevelenhancedyoloforrobustvehicledetectionintrafficsurveillance AT pengjin mmeyolomultisensormultilevelenhancedyoloforrobustvehicledetectionintrafficsurveillance AT qiminxu mmeyolomultisensormultilevelenhancedyoloforrobustvehicledetectionintrafficsurveillance AT zhengliangsun mmeyolomultisensormultilevelenhancedyoloforrobustvehicledetectionintrafficsurveillance AT xiangsong mmeyolomultisensormultilevelenhancedyoloforrobustvehicledetectionintrafficsurveillance |