A Novel Interactive Fusion Method with Images and Point Clouds for 3D Object Detection
This paper aims at tackling the task of fusion feature from images and their corresponding point clouds for 3D object detection in autonomous driving scenarios based on AVOD, an Aggregate View Object Detection network. The proposed fusion algorithms fuse features targeted from Bird’s Eye V...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2019-03-01
|
| Series: | Applied Sciences |
| Subjects: | |
| Online Access: | http://www.mdpi.com/2076-3417/9/6/1065 |
| _version_ | 1819278045107716096 |
|---|---|
| author | Kai Xu Zhile Yang Yangjie Xu Liangbing Feng |
| author_facet | Kai Xu Zhile Yang Yangjie Xu Liangbing Feng |
| author_sort | Kai Xu |
| collection | DOAJ |
| description | This paper aims at tackling the task of fusion feature from images and their corresponding point clouds for 3D object detection in autonomous driving scenarios based on AVOD, an Aggregate View Object Detection network. The proposed fusion algorithms fuse features targeted from Bird’s Eye View (BEV) LIDAR point clouds and their corresponding RGB images. Differing in existing fusion methods, which are simply the adoption of the concatenation module, the element-wise sum module or the element-wise mean module, our proposed fusion algorithms enhance the interaction between BEV feature maps and their corresponding image feature maps by designing a novel structure, where single level feature maps and utilize multilevel feature maps. Experiments show that our proposed fusion algorithm produces better results on 3D mAP and AHS with less speed loss compared to the existing fusion method used on the KITTI 3D object detection benchmark. |
| first_indexed | 2024-12-24T00:05:46Z |
| format | Article |
| id | doaj.art-881979ecffdf4530a0bf49e3a9aa1a4e |
| institution | Directory Open Access Journal |
| issn | 2076-3417 |
| language | English |
| last_indexed | 2024-12-24T00:05:46Z |
| publishDate | 2019-03-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Applied Sciences |
| spelling | doaj.art-881979ecffdf4530a0bf49e3a9aa1a4e2022-12-21T17:25:00ZengMDPI AGApplied Sciences2076-34172019-03-0196106510.3390/app9061065app9061065A Novel Interactive Fusion Method with Images and Point Clouds for 3D Object DetectionKai Xu0Zhile Yang1Yangjie Xu2Liangbing Feng3Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen 518055, ChinaShenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen 518055, ChinaShenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen 518055, ChinaShenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen 518055, ChinaThis paper aims at tackling the task of fusion feature from images and their corresponding point clouds for 3D object detection in autonomous driving scenarios based on AVOD, an Aggregate View Object Detection network. The proposed fusion algorithms fuse features targeted from Bird’s Eye View (BEV) LIDAR point clouds and their corresponding RGB images. Differing in existing fusion methods, which are simply the adoption of the concatenation module, the element-wise sum module or the element-wise mean module, our proposed fusion algorithms enhance the interaction between BEV feature maps and their corresponding image feature maps by designing a novel structure, where single level feature maps and utilize multilevel feature maps. Experiments show that our proposed fusion algorithm produces better results on 3D mAP and AHS with less speed loss compared to the existing fusion method used on the KITTI 3D object detection benchmark.http://www.mdpi.com/2076-3417/9/6/1065fusionpoint cloudsimagesobject detection |
| spellingShingle | Kai Xu Zhile Yang Yangjie Xu Liangbing Feng A Novel Interactive Fusion Method with Images and Point Clouds for 3D Object Detection Applied Sciences fusion point clouds images object detection |
| title | A Novel Interactive Fusion Method with Images and Point Clouds for 3D Object Detection |
| title_full | A Novel Interactive Fusion Method with Images and Point Clouds for 3D Object Detection |
| title_fullStr | A Novel Interactive Fusion Method with Images and Point Clouds for 3D Object Detection |
| title_full_unstemmed | A Novel Interactive Fusion Method with Images and Point Clouds for 3D Object Detection |
| title_short | A Novel Interactive Fusion Method with Images and Point Clouds for 3D Object Detection |
| title_sort | novel interactive fusion method with images and point clouds for 3d object detection |
| topic | fusion point clouds images object detection |
| url | http://www.mdpi.com/2076-3417/9/6/1065 |
| work_keys_str_mv | AT kaixu anovelinteractivefusionmethodwithimagesandpointcloudsfor3dobjectdetection AT zhileyang anovelinteractivefusionmethodwithimagesandpointcloudsfor3dobjectdetection AT yangjiexu anovelinteractivefusionmethodwithimagesandpointcloudsfor3dobjectdetection AT liangbingfeng anovelinteractivefusionmethodwithimagesandpointcloudsfor3dobjectdetection AT kaixu novelinteractivefusionmethodwithimagesandpointcloudsfor3dobjectdetection AT zhileyang novelinteractivefusionmethodwithimagesandpointcloudsfor3dobjectdetection AT yangjiexu novelinteractivefusionmethodwithimagesandpointcloudsfor3dobjectdetection AT liangbingfeng novelinteractivefusionmethodwithimagesandpointcloudsfor3dobjectdetection |