Comparison of Reinforcement Learning and Model Predictive Control for Automated Generation of Optimal Control for Dynamic Systems within a Design Space Exploration Framework
This work provides a study of methods for the automated derivation of control strategies for over-actuated systems. For this purpose, Reinforcement Learning (RL) and Model Predictive Control (MPC) approximating the solution of the Optimal Control Problem (OCP) are compared using the example of an ov...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Society of Automotive Engineers of Japan, Inc.
2024-01-01
|
| Series: | International Journal of Automotive Engineering |
| Online Access: | https://www.jstage.jst.go.jp/article/jsaeijae/15/1/15_20244099/_article/-char/ja |
| _version_ | 1797272359350042624 |
|---|---|
| author | Patrick Hoffmann Kirill Gorelik Valentin Ivanov |
| author_facet | Patrick Hoffmann Kirill Gorelik Valentin Ivanov |
| author_sort | Patrick Hoffmann |
| collection | DOAJ |
| description | This work provides a study of methods for the automated derivation of control strategies for over-actuated systems. For this purpose, Reinforcement Learning (RL) and Model Predictive Control (MPC) approximating the solution of the Optimal Control Problem (OCP) are compared using the example of an over-actuated vehicle model executing an ISO Double Lane Change (DLC). This exemplary driving maneuver is chosen due to its critical vehicle dynamics for the comparison of algorithms in terms of control performance and possible automation within a design space exploration framework. The algorithms show reasonable control results for the goal of this study, although there are differences in terms of driving stability. While Model Predictive Control first requires the optimization of the trajectory, which should then be optimally tracked, RL may combine both in one step. In addition, manual effort required to adapt the OCP problem to new design variants for solving it with RL and MPC is evaluated and assessed with respect to its automation. As a result of this study, an Actor-Critic Reinforcement Learning method is recommended for the automated derivation of control strategies in the context of a design space exploration. |
| first_indexed | 2024-03-07T14:27:11Z |
| format | Article |
| id | doaj.art-80123fe37d8047e6a33df25617201354 |
| institution | Directory Open Access Journal |
| issn | 2185-0992 |
| language | English |
| last_indexed | 2024-03-07T14:27:11Z |
| publishDate | 2024-01-01 |
| publisher | Society of Automotive Engineers of Japan, Inc. |
| record_format | Article |
| series | International Journal of Automotive Engineering |
| spelling | doaj.art-80123fe37d8047e6a33df256172013542024-03-06T07:44:10ZengSociety of Automotive Engineers of Japan, Inc.International Journal of Automotive Engineering2185-09922024-01-01151192610.20485/jsaeijae.15.1_19Comparison of Reinforcement Learning and Model Predictive Control for Automated Generation of Optimal Control for Dynamic Systems within a Design Space Exploration FrameworkPatrick Hoffmann0Kirill Gorelik1Valentin Ivanov2Robert Bosch GmbHRobert Bosch GmbHSmart Vehicle Systems - Working GroupThis work provides a study of methods for the automated derivation of control strategies for over-actuated systems. For this purpose, Reinforcement Learning (RL) and Model Predictive Control (MPC) approximating the solution of the Optimal Control Problem (OCP) are compared using the example of an over-actuated vehicle model executing an ISO Double Lane Change (DLC). This exemplary driving maneuver is chosen due to its critical vehicle dynamics for the comparison of algorithms in terms of control performance and possible automation within a design space exploration framework. The algorithms show reasonable control results for the goal of this study, although there are differences in terms of driving stability. While Model Predictive Control first requires the optimization of the trajectory, which should then be optimally tracked, RL may combine both in one step. In addition, manual effort required to adapt the OCP problem to new design variants for solving it with RL and MPC is evaluated and assessed with respect to its automation. As a result of this study, an Actor-Critic Reinforcement Learning method is recommended for the automated derivation of control strategies in the context of a design space exploration.https://www.jstage.jst.go.jp/article/jsaeijae/15/1/15_20244099/_article/-char/ja |
| spellingShingle | Patrick Hoffmann Kirill Gorelik Valentin Ivanov Comparison of Reinforcement Learning and Model Predictive Control for Automated Generation of Optimal Control for Dynamic Systems within a Design Space Exploration Framework International Journal of Automotive Engineering |
| title | Comparison of Reinforcement Learning and Model Predictive Control for Automated Generation of Optimal Control for Dynamic Systems within a Design Space Exploration Framework |
| title_full | Comparison of Reinforcement Learning and Model Predictive Control for Automated Generation of Optimal Control for Dynamic Systems within a Design Space Exploration Framework |
| title_fullStr | Comparison of Reinforcement Learning and Model Predictive Control for Automated Generation of Optimal Control for Dynamic Systems within a Design Space Exploration Framework |
| title_full_unstemmed | Comparison of Reinforcement Learning and Model Predictive Control for Automated Generation of Optimal Control for Dynamic Systems within a Design Space Exploration Framework |
| title_short | Comparison of Reinforcement Learning and Model Predictive Control for Automated Generation of Optimal Control for Dynamic Systems within a Design Space Exploration Framework |
| title_sort | comparison of reinforcement learning and model predictive control for automated generation of optimal control for dynamic systems within a design space exploration framework |
| url | https://www.jstage.jst.go.jp/article/jsaeijae/15/1/15_20244099/_article/-char/ja |
| work_keys_str_mv | AT patrickhoffmann comparisonofreinforcementlearningandmodelpredictivecontrolforautomatedgenerationofoptimalcontrolfordynamicsystemswithinadesignspaceexplorationframework AT kirillgorelik comparisonofreinforcementlearningandmodelpredictivecontrolforautomatedgenerationofoptimalcontrolfordynamicsystemswithinadesignspaceexplorationframework AT valentinivanov comparisonofreinforcementlearningandmodelpredictivecontrolforautomatedgenerationofoptimalcontrolfordynamicsystemswithinadesignspaceexplorationframework |