View-Invariant Spatiotemporal Attentive Motion Planning and Control Network for Autonomous Vehicles
Autonomous driving vehicles (ADVs) are sleeping giant intelligent machines that perceive their environment and make driving decisions. Most existing ADSs are built as hand-engineered perception-planning-control pipelines. However, designing generalized handcrafted rules for autonomous driving in an...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2022-12-01
|
| Series: | Machines |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2075-1702/10/12/1193 |
| _version_ | 1827638026069606400 |
|---|---|
| author | Melese Ayalew Shijie Zhou Imran Memon Md Belal Bin Heyat Faijan Akhtar Xiaojuan Zhang |
| author_facet | Melese Ayalew Shijie Zhou Imran Memon Md Belal Bin Heyat Faijan Akhtar Xiaojuan Zhang |
| author_sort | Melese Ayalew |
| collection | DOAJ |
| description | Autonomous driving vehicles (ADVs) are sleeping giant intelligent machines that perceive their environment and make driving decisions. Most existing ADSs are built as hand-engineered perception-planning-control pipelines. However, designing generalized handcrafted rules for autonomous driving in an urban environment is complex. An alternative approach is imitation learning (IL) from human driving demonstrations. However, most previous studies on IL for autonomous driving face several critical challenges: (1) poor generalization ability toward the unseen environment due to distribution shift problems such as changes in driving views and weather conditions; (2) lack of interpretability; and (3) mostly trained to learn the single driving task. To address these challenges, we propose a view-invariant spatiotemporal attentive planning and control network for autonomous vehicles. The proposed method first extracts spatiotemporal representations from images of a front and top driving view sequence through attentive Siamese 3DResNet. Then, the maximum mean discrepancy loss (MMD) is employed to minimize spatiotemporal discrepancies between these driving views and produce an invariant spatiotemporal representation, which reduces domain shift due to view change. Finally, the multitasking learning (MTL) method is employed to jointly train trajectory planning and high-level control tasks based on learned representations and previous motions. Results of extensive experimental evaluations on a large autonomous driving dataset with various weather/lighting conditions verified that the proposed method is effective for feasible motion planning and control in autonomous vehicles. |
| first_indexed | 2024-03-09T16:09:55Z |
| format | Article |
| id | doaj.art-4ff14249c1464851a57519076fd4d831 |
| institution | Directory Open Access Journal |
| issn | 2075-1702 |
| language | English |
| last_indexed | 2024-03-09T16:09:55Z |
| publishDate | 2022-12-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Machines |
| spelling | doaj.art-4ff14249c1464851a57519076fd4d8312023-11-24T16:17:12ZengMDPI AGMachines2075-17022022-12-011012119310.3390/machines10121193View-Invariant Spatiotemporal Attentive Motion Planning and Control Network for Autonomous VehiclesMelese Ayalew0Shijie Zhou1Imran Memon2Md Belal Bin Heyat3Faijan Akhtar4Xiaojuan Zhang5School of Information and Software Engineering, University of Electronic Science and Technology of China, Chengdu 610054, ChinaSchool of Information and Software Engineering, University of Electronic Science and Technology of China, Chengdu 610054, ChinaDepartment of Computer Science, Shahdadkot Campus, Shah Abdul Latif University, Khairpur 66111, PakistanIoT Research Center, College of Computer Science and Software Engineering, Shenzhen University, Shenzhen 518060, ChinaSchool of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu 610054, ChinaZhejiang CloudNeedle Information Technology Co., Ltd., Hangzhou 311121, ChinaAutonomous driving vehicles (ADVs) are sleeping giant intelligent machines that perceive their environment and make driving decisions. Most existing ADSs are built as hand-engineered perception-planning-control pipelines. However, designing generalized handcrafted rules for autonomous driving in an urban environment is complex. An alternative approach is imitation learning (IL) from human driving demonstrations. However, most previous studies on IL for autonomous driving face several critical challenges: (1) poor generalization ability toward the unseen environment due to distribution shift problems such as changes in driving views and weather conditions; (2) lack of interpretability; and (3) mostly trained to learn the single driving task. To address these challenges, we propose a view-invariant spatiotemporal attentive planning and control network for autonomous vehicles. The proposed method first extracts spatiotemporal representations from images of a front and top driving view sequence through attentive Siamese 3DResNet. Then, the maximum mean discrepancy loss (MMD) is employed to minimize spatiotemporal discrepancies between these driving views and produce an invariant spatiotemporal representation, which reduces domain shift due to view change. Finally, the multitasking learning (MTL) method is employed to jointly train trajectory planning and high-level control tasks based on learned representations and previous motions. Results of extensive experimental evaluations on a large autonomous driving dataset with various weather/lighting conditions verified that the proposed method is effective for feasible motion planning and control in autonomous vehicles.https://www.mdpi.com/2075-1702/10/12/1193autonomous vehiclesdeep learninginvariant representation learningmotion planningspatiotemporal attentionvehicle control |
| spellingShingle | Melese Ayalew Shijie Zhou Imran Memon Md Belal Bin Heyat Faijan Akhtar Xiaojuan Zhang View-Invariant Spatiotemporal Attentive Motion Planning and Control Network for Autonomous Vehicles Machines autonomous vehicles deep learning invariant representation learning motion planning spatiotemporal attention vehicle control |
| title | View-Invariant Spatiotemporal Attentive Motion Planning and Control Network for Autonomous Vehicles |
| title_full | View-Invariant Spatiotemporal Attentive Motion Planning and Control Network for Autonomous Vehicles |
| title_fullStr | View-Invariant Spatiotemporal Attentive Motion Planning and Control Network for Autonomous Vehicles |
| title_full_unstemmed | View-Invariant Spatiotemporal Attentive Motion Planning and Control Network for Autonomous Vehicles |
| title_short | View-Invariant Spatiotemporal Attentive Motion Planning and Control Network for Autonomous Vehicles |
| title_sort | view invariant spatiotemporal attentive motion planning and control network for autonomous vehicles |
| topic | autonomous vehicles deep learning invariant representation learning motion planning spatiotemporal attention vehicle control |
| url | https://www.mdpi.com/2075-1702/10/12/1193 |
| work_keys_str_mv | AT meleseayalew viewinvariantspatiotemporalattentivemotionplanningandcontrolnetworkforautonomousvehicles AT shijiezhou viewinvariantspatiotemporalattentivemotionplanningandcontrolnetworkforautonomousvehicles AT imranmemon viewinvariantspatiotemporalattentivemotionplanningandcontrolnetworkforautonomousvehicles AT mdbelalbinheyat viewinvariantspatiotemporalattentivemotionplanningandcontrolnetworkforautonomousvehicles AT faijanakhtar viewinvariantspatiotemporalattentivemotionplanningandcontrolnetworkforautonomousvehicles AT xiaojuanzhang viewinvariantspatiotemporalattentivemotionplanningandcontrolnetworkforautonomousvehicles |