Hierarchical reinforcement learning for self‐driving decision‐making without reliance on labelled driving data
Decision making for self‐driving cars is usually tackled by manually encoding rules from drivers’ behaviours or imitating drivers’ manipulation using supervised learning techniques. Both of them rely on mass driving data to cover all possible driving scenarios. This study presents a hierarchical rei...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Wiley
2020-05-01
|
| Series: | IET Intelligent Transport Systems |
| Subjects: | |
| Online Access: | https://doi.org/10.1049/iet-its.2019.0317 |
| _version_ | 1811188629454716928 |
|---|---|
| author | Jingliang Duan Shengbo Eben Li Yang Guan Qi Sun Bo Cheng |
| author_facet | Jingliang Duan Shengbo Eben Li Yang Guan Qi Sun Bo Cheng |
| author_sort | Jingliang Duan |
| collection | DOAJ |
| description | Decision making for self‐driving cars is usually tackled by manually encoding rules from drivers’ behaviours or imitating drivers’ manipulation using supervised learning techniques. Both of them rely on mass driving data to cover all possible driving scenarios. This study presents a hierarchical reinforcement learning method for decision making of self‐driving cars, which does not depend on a large amount of labelled driving data. This method comprehensively considers both high‐level manoeuvre selection and low‐level motion control in both lateral and longitudinal directions. The authors firstly decompose the driving tasks into three manoeuvres, including driving in lane, right lane change and left lane change, and learn the sub‐policy for each manoeuvre. Then, a master policy is learned to choose the manoeuvre policy to be executed in the current state. All policies, including master policy and manoeuvre policies, are represented by fully‐connected neural networks and trained by using asynchronous parallel reinforcement learners, which builds a mapping from the sensory outputs to driving decisions. Different state spaces and reward functions are designed for each manoeuvre. They apply this method to a highway driving scenario, which demonstrates that it can realise smooth and safe decision making for self‐driving cars. |
| first_indexed | 2024-04-11T14:23:12Z |
| format | Article |
| id | doaj.art-478d25d680c74bcdbb1cf47749f8c740 |
| institution | Directory Open Access Journal |
| issn | 1751-956X 1751-9578 |
| language | English |
| last_indexed | 2024-04-11T14:23:12Z |
| publishDate | 2020-05-01 |
| publisher | Wiley |
| record_format | Article |
| series | IET Intelligent Transport Systems |
| spelling | doaj.art-478d25d680c74bcdbb1cf47749f8c7402022-12-22T04:19:00ZengWileyIET Intelligent Transport Systems1751-956X1751-95782020-05-0114529730510.1049/iet-its.2019.0317Hierarchical reinforcement learning for self‐driving decision‐making without reliance on labelled driving dataJingliang Duan0Shengbo Eben Li1Yang Guan2Qi Sun3Bo Cheng4School of Vehicle and Mobility, Tsinghua UniversityBeijing100084People's Republic of ChinaSchool of Vehicle and Mobility, Tsinghua UniversityBeijing100084People's Republic of ChinaSchool of Vehicle and Mobility, Tsinghua UniversityBeijing100084People's Republic of ChinaSchool of Vehicle and Mobility, Tsinghua UniversityBeijing100084People's Republic of ChinaSchool of Vehicle and Mobility, Tsinghua UniversityBeijing100084People's Republic of ChinaDecision making for self‐driving cars is usually tackled by manually encoding rules from drivers’ behaviours or imitating drivers’ manipulation using supervised learning techniques. Both of them rely on mass driving data to cover all possible driving scenarios. This study presents a hierarchical reinforcement learning method for decision making of self‐driving cars, which does not depend on a large amount of labelled driving data. This method comprehensively considers both high‐level manoeuvre selection and low‐level motion control in both lateral and longitudinal directions. The authors firstly decompose the driving tasks into three manoeuvres, including driving in lane, right lane change and left lane change, and learn the sub‐policy for each manoeuvre. Then, a master policy is learned to choose the manoeuvre policy to be executed in the current state. All policies, including master policy and manoeuvre policies, are represented by fully‐connected neural networks and trained by using asynchronous parallel reinforcement learners, which builds a mapping from the sensory outputs to driving decisions. Different state spaces and reward functions are designed for each manoeuvre. They apply this method to a highway driving scenario, which demonstrates that it can realise smooth and safe decision making for self‐driving cars.https://doi.org/10.1049/iet-its.2019.0317self‐driving carslabelled driving datahigh‐level manoeuvre selectionlow‐level motion controlasynchronous parallel reinforcement learnersdriving decisions |
| spellingShingle | Jingliang Duan Shengbo Eben Li Yang Guan Qi Sun Bo Cheng Hierarchical reinforcement learning for self‐driving decision‐making without reliance on labelled driving data IET Intelligent Transport Systems self‐driving cars labelled driving data high‐level manoeuvre selection low‐level motion control asynchronous parallel reinforcement learners driving decisions |
| title | Hierarchical reinforcement learning for self‐driving decision‐making without reliance on labelled driving data |
| title_full | Hierarchical reinforcement learning for self‐driving decision‐making without reliance on labelled driving data |
| title_fullStr | Hierarchical reinforcement learning for self‐driving decision‐making without reliance on labelled driving data |
| title_full_unstemmed | Hierarchical reinforcement learning for self‐driving decision‐making without reliance on labelled driving data |
| title_short | Hierarchical reinforcement learning for self‐driving decision‐making without reliance on labelled driving data |
| title_sort | hierarchical reinforcement learning for self driving decision making without reliance on labelled driving data |
| topic | self‐driving cars labelled driving data high‐level manoeuvre selection low‐level motion control asynchronous parallel reinforcement learners driving decisions |
| url | https://doi.org/10.1049/iet-its.2019.0317 |
| work_keys_str_mv | AT jingliangduan hierarchicalreinforcementlearningforselfdrivingdecisionmakingwithoutrelianceonlabelleddrivingdata AT shengboebenli hierarchicalreinforcementlearningforselfdrivingdecisionmakingwithoutrelianceonlabelleddrivingdata AT yangguan hierarchicalreinforcementlearningforselfdrivingdecisionmakingwithoutrelianceonlabelleddrivingdata AT qisun hierarchicalreinforcementlearningforselfdrivingdecisionmakingwithoutrelianceonlabelleddrivingdata AT bocheng hierarchicalreinforcementlearningforselfdrivingdecisionmakingwithoutrelianceonlabelleddrivingdata |