GAN-Based LiDAR Translation between Sunny and Adverse Weather for Autonomous Driving and Driving Simulation
Autonomous driving requires robust and highly accurate perception technologies. Various deep learning algorithms based on only image processing satisfy this requirement, but few such algorithms are based on LiDAR. However, images are only one part of the perceptible sensors in an autonomous driving...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2022-07-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/22/14/5287 |
| _version_ | 1827604220682960896 |
|---|---|
| author | Jinho Lee Daiki Shiotsuka Toshiaki Nishimori Kenta Nakao Shunsuke Kamijo |
| author_facet | Jinho Lee Daiki Shiotsuka Toshiaki Nishimori Kenta Nakao Shunsuke Kamijo |
| author_sort | Jinho Lee |
| collection | DOAJ |
| description | Autonomous driving requires robust and highly accurate perception technologies. Various deep learning algorithms based on only image processing satisfy this requirement, but few such algorithms are based on LiDAR. However, images are only one part of the perceptible sensors in an autonomous driving vehicle; LiDAR is also essential for the recognition of driving environments. The main reason why there exist few deep learning algorithms based on LiDAR is a lack of data. Recent translation technology using generative adversarial networks (GANs) has been proposed to deal with this problem. However, these technologies focus on only image-to-image translation, although a lack of data occurs more often with LiDAR than with images. LiDAR translation technology is required not only for data augmentation, but also for driving simulation, which allows algorithms to practice driving as if they were commanding a real vehicle, before doing so in the real world. In other words, driving simulation is a key technology for evaluating and verifying algorithms which are practically applied to vehicles. In this paper, we propose a GAN-based LiDAR translation algorithm for autonomous driving and driving simulation. It is the first LiDAR translation approach that can deal with various types of weather that are based on an empirical approach. We tested the proposed method on the JARI data set, which was collected under various adverse weather scenarios with diverse precipitation and visible distance settings. The proposed method was also applied to the real-world Spain data set. Our experimental results demonstrate that the proposed method can generate realistic LiDAR data under adverse weather conditions. |
| first_indexed | 2024-03-09T05:55:57Z |
| format | Article |
| id | doaj.art-1563d6eccc3a420ba68c1ccdc16a712a |
| institution | Directory Open Access Journal |
| issn | 1424-8220 |
| language | English |
| last_indexed | 2024-03-09T05:55:57Z |
| publishDate | 2022-07-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj.art-1563d6eccc3a420ba68c1ccdc16a712a2023-12-03T12:13:00ZengMDPI AGSensors1424-82202022-07-012214528710.3390/s22145287GAN-Based LiDAR Translation between Sunny and Adverse Weather for Autonomous Driving and Driving SimulationJinho Lee0Daiki Shiotsuka1Toshiaki Nishimori2Kenta Nakao3Shunsuke Kamijo4Emerging Design and Informatics Course, Graduate School of Interdisciplinary Information Studies, The University of Tokyo, 4 Chome-6-1 Komaba, Meguro City, Tokyo 153-0041, JapanEmerging Design and Informatics Course, Graduate School of Interdisciplinary Information Studies, The University of Tokyo, 4 Chome-6-1 Komaba, Meguro City, Tokyo 153-0041, JapanMitsubishi Heavy Industries Machinery Systems Ltd., 1-1, Wadasaki-cho 1-chome, Hyogo-ku, Kobe 652-8585, JapanMitsubishi Heavy Industries Ltd., 1-1, Wadasaki-cho 1-chome, Hyogo-ku, Kobe 652-8585, JapanThe Institute of Industrial Science (IIS), The University of Tokyo, 4 Chome-6-1 Komaba, Meguro City, Tokyo 153-0041, JapanAutonomous driving requires robust and highly accurate perception technologies. Various deep learning algorithms based on only image processing satisfy this requirement, but few such algorithms are based on LiDAR. However, images are only one part of the perceptible sensors in an autonomous driving vehicle; LiDAR is also essential for the recognition of driving environments. The main reason why there exist few deep learning algorithms based on LiDAR is a lack of data. Recent translation technology using generative adversarial networks (GANs) has been proposed to deal with this problem. However, these technologies focus on only image-to-image translation, although a lack of data occurs more often with LiDAR than with images. LiDAR translation technology is required not only for data augmentation, but also for driving simulation, which allows algorithms to practice driving as if they were commanding a real vehicle, before doing so in the real world. In other words, driving simulation is a key technology for evaluating and verifying algorithms which are practically applied to vehicles. In this paper, we propose a GAN-based LiDAR translation algorithm for autonomous driving and driving simulation. It is the first LiDAR translation approach that can deal with various types of weather that are based on an empirical approach. We tested the proposed method on the JARI data set, which was collected under various adverse weather scenarios with diverse precipitation and visible distance settings. The proposed method was also applied to the real-world Spain data set. Our experimental results demonstrate that the proposed method can generate realistic LiDAR data under adverse weather conditions.https://www.mdpi.com/1424-8220/22/14/5287LiDAR-to-LiDAR translationadverse weatherautonomous drivingdriving simulatordata augmentationgenerative adversarial networks |
| spellingShingle | Jinho Lee Daiki Shiotsuka Toshiaki Nishimori Kenta Nakao Shunsuke Kamijo GAN-Based LiDAR Translation between Sunny and Adverse Weather for Autonomous Driving and Driving Simulation Sensors LiDAR-to-LiDAR translation adverse weather autonomous driving driving simulator data augmentation generative adversarial networks |
| title | GAN-Based LiDAR Translation between Sunny and Adverse Weather for Autonomous Driving and Driving Simulation |
| title_full | GAN-Based LiDAR Translation between Sunny and Adverse Weather for Autonomous Driving and Driving Simulation |
| title_fullStr | GAN-Based LiDAR Translation between Sunny and Adverse Weather for Autonomous Driving and Driving Simulation |
| title_full_unstemmed | GAN-Based LiDAR Translation between Sunny and Adverse Weather for Autonomous Driving and Driving Simulation |
| title_short | GAN-Based LiDAR Translation between Sunny and Adverse Weather for Autonomous Driving and Driving Simulation |
| title_sort | gan based lidar translation between sunny and adverse weather for autonomous driving and driving simulation |
| topic | LiDAR-to-LiDAR translation adverse weather autonomous driving driving simulator data augmentation generative adversarial networks |
| url | https://www.mdpi.com/1424-8220/22/14/5287 |
| work_keys_str_mv | AT jinholee ganbasedlidartranslationbetweensunnyandadverseweatherforautonomousdrivinganddrivingsimulation AT daikishiotsuka ganbasedlidartranslationbetweensunnyandadverseweatherforautonomousdrivinganddrivingsimulation AT toshiakinishimori ganbasedlidartranslationbetweensunnyandadverseweatherforautonomousdrivinganddrivingsimulation AT kentanakao ganbasedlidartranslationbetweensunnyandadverseweatherforautonomousdrivinganddrivingsimulation AT shunsukekamijo ganbasedlidartranslationbetweensunnyandadverseweatherforautonomousdrivinganddrivingsimulation |