Multiple Event-Based Simulation Scenario Generation Approach for Autonomous Vehicle Smart Sensors and Devices
Nowadays, deep learning methods based on a virtual environment are widely applied to research and technology development for autonomous vehicle’s smart sensors and devices. Learning various driving environments in advance is important to handle unexpected situations that can exist in the r...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2019-10-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/19/20/4456 |
| _version_ | 1798004105375907840 |
|---|---|
| author | Jisun Park Mingyun Wen Yunsick Sung Kyungeun Cho |
| author_facet | Jisun Park Mingyun Wen Yunsick Sung Kyungeun Cho |
| author_sort | Jisun Park |
| collection | DOAJ |
| description | Nowadays, deep learning methods based on a virtual environment are widely applied to research and technology development for autonomous vehicle’s smart sensors and devices. Learning various driving environments in advance is important to handle unexpected situations that can exist in the real world and to continue driving without accident. For training smart sensors and devices of an autonomous vehicle well, a virtual simulator should create scenarios of various possible real-world situations. To create reality-based scenarios, data on the real environment must be collected from a real driving vehicle or a scenario analysis process conducted by experts. However, these two approaches increase the period and the cost of scenario generation as more scenarios are created. This paper proposes a scenario generation method based on deep learning to create scenarios automatically for training autonomous vehicle smart sensors and devices. To generate various scenarios, the proposed method extracts multiple events from a video which is taken on a real road by using deep learning and generates the multiple event in a virtual simulator. First, Faster-region based convolution neural network (Faster-RCNN) extracts bounding boxes of each object in a driving video. Second, the high-level event bounding boxes are calculated. Third, long-term recurrent convolution networks (LRCN) classify each type of extracted event. Finally, all multiple event classification results are combined into one scenario. The generated scenarios can be used in an autonomous driving simulator to teach multiple events that occur during real-world driving. To verify the performance of the proposed scenario generation method, experiments using real driving video data and a virtual simulator were conducted. The results for deep learning model show an accuracy of 95.6%; furthermore, multiple high-level events were extracted, and various scenarios were generated in a virtual simulator for smart sensors and devices of an autonomous vehicle. |
| first_indexed | 2024-04-11T12:19:22Z |
| format | Article |
| id | doaj.art-7ce7f24938a64d48834b71a0af9bbf95 |
| institution | Directory Open Access Journal |
| issn | 1424-8220 |
| language | English |
| last_indexed | 2024-04-11T12:19:22Z |
| publishDate | 2019-10-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj.art-7ce7f24938a64d48834b71a0af9bbf952022-12-22T04:24:10ZengMDPI AGSensors1424-82202019-10-011920445610.3390/s19204456s19204456Multiple Event-Based Simulation Scenario Generation Approach for Autonomous Vehicle Smart Sensors and DevicesJisun Park0Mingyun Wen1Yunsick Sung2Kyungeun Cho3Department of Multimedia Engineering, Dongguk University-Seoul, Seoul 04620, KoreaDepartment of Multimedia Engineering, Dongguk University-Seoul, Seoul 04620, KoreaDepartment of Multimedia Engineering, Dongguk University-Seoul, Seoul 04620, KoreaDepartment of Multimedia Engineering, Dongguk University-Seoul, Seoul 04620, KoreaNowadays, deep learning methods based on a virtual environment are widely applied to research and technology development for autonomous vehicle’s smart sensors and devices. Learning various driving environments in advance is important to handle unexpected situations that can exist in the real world and to continue driving without accident. For training smart sensors and devices of an autonomous vehicle well, a virtual simulator should create scenarios of various possible real-world situations. To create reality-based scenarios, data on the real environment must be collected from a real driving vehicle or a scenario analysis process conducted by experts. However, these two approaches increase the period and the cost of scenario generation as more scenarios are created. This paper proposes a scenario generation method based on deep learning to create scenarios automatically for training autonomous vehicle smart sensors and devices. To generate various scenarios, the proposed method extracts multiple events from a video which is taken on a real road by using deep learning and generates the multiple event in a virtual simulator. First, Faster-region based convolution neural network (Faster-RCNN) extracts bounding boxes of each object in a driving video. Second, the high-level event bounding boxes are calculated. Third, long-term recurrent convolution networks (LRCN) classify each type of extracted event. Finally, all multiple event classification results are combined into one scenario. The generated scenarios can be used in an autonomous driving simulator to teach multiple events that occur during real-world driving. To verify the performance of the proposed scenario generation method, experiments using real driving video data and a virtual simulator were conducted. The results for deep learning model show an accuracy of 95.6%; furthermore, multiple high-level events were extracted, and various scenarios were generated in a virtual simulator for smart sensors and devices of an autonomous vehicle.https://www.mdpi.com/1424-8220/19/20/4456scenario generationautonomous vehiclesmart sensor and devicedeep learning |
| spellingShingle | Jisun Park Mingyun Wen Yunsick Sung Kyungeun Cho Multiple Event-Based Simulation Scenario Generation Approach for Autonomous Vehicle Smart Sensors and Devices Sensors scenario generation autonomous vehicle smart sensor and device deep learning |
| title | Multiple Event-Based Simulation Scenario Generation Approach for Autonomous Vehicle Smart Sensors and Devices |
| title_full | Multiple Event-Based Simulation Scenario Generation Approach for Autonomous Vehicle Smart Sensors and Devices |
| title_fullStr | Multiple Event-Based Simulation Scenario Generation Approach for Autonomous Vehicle Smart Sensors and Devices |
| title_full_unstemmed | Multiple Event-Based Simulation Scenario Generation Approach for Autonomous Vehicle Smart Sensors and Devices |
| title_short | Multiple Event-Based Simulation Scenario Generation Approach for Autonomous Vehicle Smart Sensors and Devices |
| title_sort | multiple event based simulation scenario generation approach for autonomous vehicle smart sensors and devices |
| topic | scenario generation autonomous vehicle smart sensor and device deep learning |
| url | https://www.mdpi.com/1424-8220/19/20/4456 |
| work_keys_str_mv | AT jisunpark multipleeventbasedsimulationscenariogenerationapproachforautonomousvehiclesmartsensorsanddevices AT mingyunwen multipleeventbasedsimulationscenariogenerationapproachforautonomousvehiclesmartsensorsanddevices AT yunsicksung multipleeventbasedsimulationscenariogenerationapproachforautonomousvehiclesmartsensorsanddevices AT kyungeuncho multipleeventbasedsimulationscenariogenerationapproachforautonomousvehiclesmartsensorsanddevices |