Volume-weighted Bellman error method for adaptive meshing in approximate dynamic programming
Optimal control and reinforcement learning have an associate “value function” which must be suitably approximated. Value function approximation problems usually have different precision requirements in different regions of the state space. An uniform gridding wastes resources in regions in which the...
Main Authors: | , |
---|---|
Format: | Article |
Language: | Spanish |
Published: |
Universitat Politecnica de Valencia
2021-12-01
|
Series: | Revista Iberoamericana de Automática e Informática Industrial RIAI |
Subjects: | |
Online Access: | https://polipapers.upv.es/index.php/RIAI/article/view/15698 |
_version_ | 1819004823562878976 |
---|---|
author | Leopoldo Armesto Antonio Sala |
author_facet | Leopoldo Armesto Antonio Sala |
author_sort | Leopoldo Armesto |
collection | DOAJ |
description | Optimal control and reinforcement learning have an associate “value function” which must be suitably approximated. Value function approximation problems usually have different precision requirements in different regions of the state space. An uniform gridding wastes resources in regions in which the value function is smooth, and, on the other hand, has not enough resolution in zones with abrupt changes. The present work proposes an adaptive meshing methodology in order to adapt to these changing requirements without incrementing too much the number of parameters of the approximator. The proposal is based on simplicial meshes and Bellman error, with a criteria to add and remove points from the mesh: modifications to proposals in earlier literature including the volume of the affected simplices are proposed, alongside with methods to manipulate the mesh triangulation. |
first_indexed | 2024-12-20T23:43:01Z |
format | Article |
id | doaj.art-6df5d07f94f74ae38836dd43f03c3e8b |
institution | Directory Open Access Journal |
issn | 1697-7912 1697-7920 |
language | Spanish |
last_indexed | 2024-12-20T23:43:01Z |
publishDate | 2021-12-01 |
publisher | Universitat Politecnica de Valencia |
record_format | Article |
series | Revista Iberoamericana de Automática e Informática Industrial RIAI |
spelling | doaj.art-6df5d07f94f74ae38836dd43f03c3e8b2022-12-21T19:23:01ZspaUniversitat Politecnica de ValenciaRevista Iberoamericana de Automática e Informática Industrial RIAI1697-79121697-79202021-12-01191374710.4995/riai.2021.156988920Volume-weighted Bellman error method for adaptive meshing in approximate dynamic programmingLeopoldo Armesto0Antonio Sala1Universitat Politècnica de ValènciaUniversitat Politècnica de ValènciaOptimal control and reinforcement learning have an associate “value function” which must be suitably approximated. Value function approximation problems usually have different precision requirements in different regions of the state space. An uniform gridding wastes resources in regions in which the value function is smooth, and, on the other hand, has not enough resolution in zones with abrupt changes. The present work proposes an adaptive meshing methodology in order to adapt to these changing requirements without incrementing too much the number of parameters of the approximator. The proposal is based on simplicial meshes and Bellman error, with a criteria to add and remove points from the mesh: modifications to proposals in earlier literature including the volume of the affected simplices are proposed, alongside with methods to manipulate the mesh triangulation.https://polipapers.upv.es/index.php/RIAI/article/view/15698control inteligenteprogramación dinámica aproximadacontrol óptimoaprendizaje |
spellingShingle | Leopoldo Armesto Antonio Sala Volume-weighted Bellman error method for adaptive meshing in approximate dynamic programming Revista Iberoamericana de Automática e Informática Industrial RIAI control inteligente programación dinámica aproximada control óptimo aprendizaje |
title | Volume-weighted Bellman error method for adaptive meshing in approximate dynamic programming |
title_full | Volume-weighted Bellman error method for adaptive meshing in approximate dynamic programming |
title_fullStr | Volume-weighted Bellman error method for adaptive meshing in approximate dynamic programming |
title_full_unstemmed | Volume-weighted Bellman error method for adaptive meshing in approximate dynamic programming |
title_short | Volume-weighted Bellman error method for adaptive meshing in approximate dynamic programming |
title_sort | volume weighted bellman error method for adaptive meshing in approximate dynamic programming |
topic | control inteligente programación dinámica aproximada control óptimo aprendizaje |
url | https://polipapers.upv.es/index.php/RIAI/article/view/15698 |
work_keys_str_mv | AT leopoldoarmesto volumeweightedbellmanerrormethodforadaptivemeshinginapproximatedynamicprogramming AT antoniosala volumeweightedbellmanerrormethodforadaptivemeshinginapproximatedynamicprogramming |