| Summary: | Performant and efficient 3D object detection and 3D semantic segmentation models for scene understanding are crucial for autonomous vehicles safety. Recent advancement in fusion methods for object detection complemented lidar’s geometric and spatial features with rich semantic information from images. On the other hand, research on semantic segmentation have mainly relied on lidar based methods for classification of points. Object predictions from the object detection pipeline could be used to localize semantic and lidar features from the detection branch to share with the semantic segmentation branch through modality fusion. Particularly, the use of transformers, which have had success with modality fusion, is explored in this project. The transformer-based fuser is about 30x smaller than the main segmentation model used, and achieved comparable performance on features extracted from the networks.
|
|---|