Multimodal Semantic Segmentation in Autonomous Driving: A Review of Current Approaches and Future Perspectives
The perception of the surrounding environment is a key requirement for autonomous driving systems, yet the computation of an accurate semantic representation of the scene starting from RGB information alone is very challenging. In particular, the lack of geometric information and the strong dependen...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2022-07-01
|
| Series: | Technologies |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2227-7080/10/4/90 |
| _version_ | 1797441598452137984 |
|---|---|
| author | Giulia Rizzoli Francesco Barbato Pietro Zanuttigh |
| author_facet | Giulia Rizzoli Francesco Barbato Pietro Zanuttigh |
| author_sort | Giulia Rizzoli |
| collection | DOAJ |
| description | The perception of the surrounding environment is a key requirement for autonomous driving systems, yet the computation of an accurate semantic representation of the scene starting from RGB information alone is very challenging. In particular, the lack of geometric information and the strong dependence on weather and illumination conditions introduce critical challenges for approaches tackling this task. For this reason, most autonomous cars exploit a variety of sensors, including color, depth or thermal cameras, LiDARs, and RADARs. How to efficiently combine all these sources of information to compute an accurate semantic description of the scene is still an unsolved task, leading to an active research field. In this survey, we start by presenting the most commonly employed acquisition setups and datasets. Then we review several different deep learning architectures for multimodal semantic segmentation. We will discuss the various techniques to combine color, depth, LiDAR, and other modalities of data at different stages of the learning architectures, and we will show how smart fusion strategies allow us to improve performances with respect to the exploitation of a single source of information. |
| first_indexed | 2024-03-09T12:25:28Z |
| format | Article |
| id | doaj.art-1e7db6c12d9141cc9e56a6233ad8115d |
| institution | Directory Open Access Journal |
| issn | 2227-7080 |
| language | English |
| last_indexed | 2024-03-09T12:25:28Z |
| publishDate | 2022-07-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Technologies |
| spelling | doaj.art-1e7db6c12d9141cc9e56a6233ad8115d2023-11-30T22:35:01ZengMDPI AGTechnologies2227-70802022-07-011049010.3390/technologies10040090Multimodal Semantic Segmentation in Autonomous Driving: A Review of Current Approaches and Future PerspectivesGiulia Rizzoli0Francesco Barbato1Pietro Zanuttigh2Department of Information Engineering, University of Padova, Via Gradenigo 6/A, 35131 Padova, ItalyDepartment of Information Engineering, University of Padova, Via Gradenigo 6/A, 35131 Padova, ItalyDepartment of Information Engineering, University of Padova, Via Gradenigo 6/A, 35131 Padova, ItalyThe perception of the surrounding environment is a key requirement for autonomous driving systems, yet the computation of an accurate semantic representation of the scene starting from RGB information alone is very challenging. In particular, the lack of geometric information and the strong dependence on weather and illumination conditions introduce critical challenges for approaches tackling this task. For this reason, most autonomous cars exploit a variety of sensors, including color, depth or thermal cameras, LiDARs, and RADARs. How to efficiently combine all these sources of information to compute an accurate semantic description of the scene is still an unsolved task, leading to an active research field. In this survey, we start by presenting the most commonly employed acquisition setups and datasets. Then we review several different deep learning architectures for multimodal semantic segmentation. We will discuss the various techniques to combine color, depth, LiDAR, and other modalities of data at different stages of the learning architectures, and we will show how smart fusion strategies allow us to improve performances with respect to the exploitation of a single source of information.https://www.mdpi.com/2227-7080/10/4/90semantic segmentationautonomous drivingmultimodalLiDARdepthmodality fusion |
| spellingShingle | Giulia Rizzoli Francesco Barbato Pietro Zanuttigh Multimodal Semantic Segmentation in Autonomous Driving: A Review of Current Approaches and Future Perspectives Technologies semantic segmentation autonomous driving multimodal LiDAR depth modality fusion |
| title | Multimodal Semantic Segmentation in Autonomous Driving: A Review of Current Approaches and Future Perspectives |
| title_full | Multimodal Semantic Segmentation in Autonomous Driving: A Review of Current Approaches and Future Perspectives |
| title_fullStr | Multimodal Semantic Segmentation in Autonomous Driving: A Review of Current Approaches and Future Perspectives |
| title_full_unstemmed | Multimodal Semantic Segmentation in Autonomous Driving: A Review of Current Approaches and Future Perspectives |
| title_short | Multimodal Semantic Segmentation in Autonomous Driving: A Review of Current Approaches and Future Perspectives |
| title_sort | multimodal semantic segmentation in autonomous driving a review of current approaches and future perspectives |
| topic | semantic segmentation autonomous driving multimodal LiDAR depth modality fusion |
| url | https://www.mdpi.com/2227-7080/10/4/90 |
| work_keys_str_mv | AT giuliarizzoli multimodalsemanticsegmentationinautonomousdrivingareviewofcurrentapproachesandfutureperspectives AT francescobarbato multimodalsemanticsegmentationinautonomousdrivingareviewofcurrentapproachesandfutureperspectives AT pietrozanuttigh multimodalsemanticsegmentationinautonomousdrivingareviewofcurrentapproachesandfutureperspectives |