Recursive State-Value Function: A Method to Reduce the Complexity of Online Computation of Dynamic Programming
This paper proposed a method to reduce the computation quantity of dynamic programming to make the time consumption be acceptable for on-line control. The proposed method is the combination of model predictive control and state-value function. This method consist of two parts, the off-line part and...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2020-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/9023935/ |
| _version_ | 1828735641229197312 |
|---|---|
| author | Wei Liao Xiaohui Wei Jizhou Lai Hao Sun |
| author_facet | Wei Liao Xiaohui Wei Jizhou Lai Hao Sun |
| author_sort | Wei Liao |
| collection | DOAJ |
| description | This paper proposed a method to reduce the computation quantity of dynamic programming to make the time consumption be acceptable for on-line control. The proposed method is the combination of model predictive control and state-value function. This method consist of two parts, the off-line part and the on-line part, where the former part is to generate an approximation of <inline-formula> <tex-math notation="LaTeX">$k$ </tex-math></inline-formula>-step recursive state-value function which represents the cumulative reward from a state in <inline-formula> <tex-math notation="LaTeX">$k$ </tex-math></inline-formula> steps under the optimal control policy, and the latter part is to work out the best action in real time using the <inline-formula> <tex-math notation="LaTeX">$k$ </tex-math></inline-formula>-step recursive state-value function both individually and in combination with MPC. At the end of this paper, some numerical examples are taken to illustrate the effectiveness of our method. Results show that compared to model predictive control and deep Q-learning, our method has some superiority. |
| first_indexed | 2024-04-12T23:11:00Z |
| format | Article |
| id | doaj.art-cc7fed4a50bd49af860da4f6375c117c |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-04-12T23:11:00Z |
| publishDate | 2020-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-cc7fed4a50bd49af860da4f6375c117c2022-12-22T03:12:49ZengIEEEIEEE Access2169-35362020-01-018611246113010.1109/ACCESS.2020.29782549023935Recursive State-Value Function: A Method to Reduce the Complexity of Online Computation of Dynamic ProgrammingWei Liao0https://orcid.org/0000-0003-2778-0572Xiaohui Wei1Jizhou Lai2Hao Sun3Key Laboratory of Fundamental Science for National Defense-Advanced Design Technology of Flight Vehicle, Nanjing University of Aeronautics and Astronautics, Nanjing, ChinaState Key Laboratory of Mechanics and Control of Mechanical Structures, Nanjing University of Aeronautics and Astronautics, Nanjing, ChinaCollege of Automation Engineering, Nanjing University of Aeronautics and Astronautics, Nanjing, ChinaKey Laboratory of Fundamental Science for National Defense-Advanced Design Technology of Flight Vehicle, Nanjing University of Aeronautics and Astronautics, Nanjing, ChinaThis paper proposed a method to reduce the computation quantity of dynamic programming to make the time consumption be acceptable for on-line control. The proposed method is the combination of model predictive control and state-value function. This method consist of two parts, the off-line part and the on-line part, where the former part is to generate an approximation of <inline-formula> <tex-math notation="LaTeX">$k$ </tex-math></inline-formula>-step recursive state-value function which represents the cumulative reward from a state in <inline-formula> <tex-math notation="LaTeX">$k$ </tex-math></inline-formula> steps under the optimal control policy, and the latter part is to work out the best action in real time using the <inline-formula> <tex-math notation="LaTeX">$k$ </tex-math></inline-formula>-step recursive state-value function both individually and in combination with MPC. At the end of this paper, some numerical examples are taken to illustrate the effectiveness of our method. Results show that compared to model predictive control and deep Q-learning, our method has some superiority.https://ieeexplore.ieee.org/document/9023935/Dynamic programmingmodel predictive controlstate-value function |
| spellingShingle | Wei Liao Xiaohui Wei Jizhou Lai Hao Sun Recursive State-Value Function: A Method to Reduce the Complexity of Online Computation of Dynamic Programming IEEE Access Dynamic programming model predictive control state-value function |
| title | Recursive State-Value Function: A Method to Reduce the Complexity of Online Computation of Dynamic Programming |
| title_full | Recursive State-Value Function: A Method to Reduce the Complexity of Online Computation of Dynamic Programming |
| title_fullStr | Recursive State-Value Function: A Method to Reduce the Complexity of Online Computation of Dynamic Programming |
| title_full_unstemmed | Recursive State-Value Function: A Method to Reduce the Complexity of Online Computation of Dynamic Programming |
| title_short | Recursive State-Value Function: A Method to Reduce the Complexity of Online Computation of Dynamic Programming |
| title_sort | recursive state value function a method to reduce the complexity of online computation of dynamic programming |
| topic | Dynamic programming model predictive control state-value function |
| url | https://ieeexplore.ieee.org/document/9023935/ |
| work_keys_str_mv | AT weiliao recursivestatevaluefunctionamethodtoreducethecomplexityofonlinecomputationofdynamicprogramming AT xiaohuiwei recursivestatevaluefunctionamethodtoreducethecomplexityofonlinecomputationofdynamicprogramming AT jizhoulai recursivestatevaluefunctionamethodtoreducethecomplexityofonlinecomputationofdynamicprogramming AT haosun recursivestatevaluefunctionamethodtoreducethecomplexityofonlinecomputationofdynamicprogramming |