Value iteration with deep neural networks for optimal control of input-affine nonlinear systems
This paper proposes a new algorithm with deep neural networks to solve optimal control problems for continuous-time input nonlinear systems based on a value iteration algorithm. The proposed algorithm applies the networks to approximating the value functions and control inputs in the iterations. Con...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Taylor & Francis Group
2021-01-01
|
| Series: | SICE Journal of Control, Measurement, and System Integration |
| Subjects: | |
| Online Access: | http://dx.doi.org/10.1080/18824889.2021.1936817 |
| _version_ | 1797661145756073984 |
|---|---|
| author | Hirofumi Beppu Ichiro Maruta Kenji Fujimoto |
| author_facet | Hirofumi Beppu Ichiro Maruta Kenji Fujimoto |
| author_sort | Hirofumi Beppu |
| collection | DOAJ |
| description | This paper proposes a new algorithm with deep neural networks to solve optimal control problems for continuous-time input nonlinear systems based on a value iteration algorithm. The proposed algorithm applies the networks to approximating the value functions and control inputs in the iterations. Consequently, the partial differential equations of the original algorithm reduce to the optimization problems for the parameters of the networks. Although the conventional algorithm can obtain the optimal control with iterative computations, each of the computations needs to be completed precisely, and it is hard to achieve sufficient precision in practice. Instead, the proposed method provides a practical method using deep neural networks and overcomes the difficulty based on a property of the networks, under which our convergence analysis shows that the proposed algorithm can achieve the minimum of the value function and the corresponding optimal controller. The effectiveness of the proposed method even with reasonable computational resources is demonstrated in two numerical simulations. |
| first_indexed | 2024-03-11T18:40:11Z |
| format | Article |
| id | doaj.art-d2c9566ba8104edf89cfc93e9b94c071 |
| institution | Directory Open Access Journal |
| issn | 1884-9970 |
| language | English |
| last_indexed | 2024-03-11T18:40:11Z |
| publishDate | 2021-01-01 |
| publisher | Taylor & Francis Group |
| record_format | Article |
| series | SICE Journal of Control, Measurement, and System Integration |
| spelling | doaj.art-d2c9566ba8104edf89cfc93e9b94c0712023-10-12T13:43:51ZengTaylor & Francis GroupSICE Journal of Control, Measurement, and System Integration1884-99702021-01-0114114014910.1080/18824889.2021.19368171936817Value iteration with deep neural networks for optimal control of input-affine nonlinear systemsHirofumi Beppu0Ichiro Maruta1Kenji Fujimoto2Department of Aeronautics and Astronautics, Graduate School of Engineering, Kyoto UniversityDepartment of Aeronautics and Astronautics, Graduate School of Engineering, Kyoto UniversityDepartment of Aeronautics and Astronautics, Graduate School of Engineering, Kyoto UniversityThis paper proposes a new algorithm with deep neural networks to solve optimal control problems for continuous-time input nonlinear systems based on a value iteration algorithm. The proposed algorithm applies the networks to approximating the value functions and control inputs in the iterations. Consequently, the partial differential equations of the original algorithm reduce to the optimization problems for the parameters of the networks. Although the conventional algorithm can obtain the optimal control with iterative computations, each of the computations needs to be completed precisely, and it is hard to achieve sufficient precision in practice. Instead, the proposed method provides a practical method using deep neural networks and overcomes the difficulty based on a property of the networks, under which our convergence analysis shows that the proposed algorithm can achieve the minimum of the value function and the corresponding optimal controller. The effectiveness of the proposed method even with reasonable computational resources is demonstrated in two numerical simulations.http://dx.doi.org/10.1080/18824889.2021.1936817value iterationoptimal controldeep neural networksinput-affine nonlinear systemsconvergence analysis |
| spellingShingle | Hirofumi Beppu Ichiro Maruta Kenji Fujimoto Value iteration with deep neural networks for optimal control of input-affine nonlinear systems SICE Journal of Control, Measurement, and System Integration value iteration optimal control deep neural networks input-affine nonlinear systems convergence analysis |
| title | Value iteration with deep neural networks for optimal control of input-affine nonlinear systems |
| title_full | Value iteration with deep neural networks for optimal control of input-affine nonlinear systems |
| title_fullStr | Value iteration with deep neural networks for optimal control of input-affine nonlinear systems |
| title_full_unstemmed | Value iteration with deep neural networks for optimal control of input-affine nonlinear systems |
| title_short | Value iteration with deep neural networks for optimal control of input-affine nonlinear systems |
| title_sort | value iteration with deep neural networks for optimal control of input affine nonlinear systems |
| topic | value iteration optimal control deep neural networks input-affine nonlinear systems convergence analysis |
| url | http://dx.doi.org/10.1080/18824889.2021.1936817 |
| work_keys_str_mv | AT hirofumibeppu valueiterationwithdeepneuralnetworksforoptimalcontrolofinputaffinenonlinearsystems AT ichiromaruta valueiterationwithdeepneuralnetworksforoptimalcontrolofinputaffinenonlinearsystems AT kenjifujimoto valueiterationwithdeepneuralnetworksforoptimalcontrolofinputaffinenonlinearsystems |