Volume-weighted Bellman error method for adaptive meshing in approximate dynamic programming

Optimal control and reinforcement learning have an associate “value function” which must be suitably approximated. Value function approximation problems usually have different precision requirements in different regions of the state space. An uniform gridding wastes resources in regions in which the...

Full description

Bibliographic Details
Main Authors: Leopoldo Armesto, Antonio Sala
Format: Article
Language:Spanish
Published: Universitat Politecnica de Valencia 2021-12-01
Series:Revista Iberoamericana de Automática e Informática Industrial RIAI
Subjects:
Online Access:https://polipapers.upv.es/index.php/RIAI/article/view/15698
_version_ 1819004823562878976
author Leopoldo Armesto
Antonio Sala
author_facet Leopoldo Armesto
Antonio Sala
author_sort Leopoldo Armesto
collection DOAJ
description Optimal control and reinforcement learning have an associate “value function” which must be suitably approximated. Value function approximation problems usually have different precision requirements in different regions of the state space. An uniform gridding wastes resources in regions in which the value function is smooth, and, on the other hand, has not enough resolution in zones with abrupt changes. The present work proposes an adaptive meshing methodology in order to adapt to these changing requirements without incrementing too much the number of parameters of the approximator. The proposal is based on simplicial meshes and Bellman error, with a criteria to add and remove points from the mesh: modifications to proposals in earlier literature including the volume of the affected simplices are proposed, alongside with methods to manipulate the mesh triangulation.
first_indexed 2024-12-20T23:43:01Z
format Article
id doaj.art-6df5d07f94f74ae38836dd43f03c3e8b
institution Directory Open Access Journal
issn 1697-7912
1697-7920
language Spanish
last_indexed 2024-12-20T23:43:01Z
publishDate 2021-12-01
publisher Universitat Politecnica de Valencia
record_format Article
series Revista Iberoamericana de Automática e Informática Industrial RIAI
spelling doaj.art-6df5d07f94f74ae38836dd43f03c3e8b2022-12-21T19:23:01ZspaUniversitat Politecnica de ValenciaRevista Iberoamericana de Automática e Informática Industrial RIAI1697-79121697-79202021-12-01191374710.4995/riai.2021.156988920Volume-weighted Bellman error method for adaptive meshing in approximate dynamic programmingLeopoldo Armesto0Antonio Sala1Universitat Politècnica de ValènciaUniversitat Politècnica de ValènciaOptimal control and reinforcement learning have an associate “value function” which must be suitably approximated. Value function approximation problems usually have different precision requirements in different regions of the state space. An uniform gridding wastes resources in regions in which the value function is smooth, and, on the other hand, has not enough resolution in zones with abrupt changes. The present work proposes an adaptive meshing methodology in order to adapt to these changing requirements without incrementing too much the number of parameters of the approximator. The proposal is based on simplicial meshes and Bellman error, with a criteria to add and remove points from the mesh: modifications to proposals in earlier literature including the volume of the affected simplices are proposed, alongside with methods to manipulate the mesh triangulation.https://polipapers.upv.es/index.php/RIAI/article/view/15698control inteligenteprogramación dinámica aproximadacontrol óptimoaprendizaje
spellingShingle Leopoldo Armesto
Antonio Sala
Volume-weighted Bellman error method for adaptive meshing in approximate dynamic programming
Revista Iberoamericana de Automática e Informática Industrial RIAI
control inteligente
programación dinámica aproximada
control óptimo
aprendizaje
title Volume-weighted Bellman error method for adaptive meshing in approximate dynamic programming
title_full Volume-weighted Bellman error method for adaptive meshing in approximate dynamic programming
title_fullStr Volume-weighted Bellman error method for adaptive meshing in approximate dynamic programming
title_full_unstemmed Volume-weighted Bellman error method for adaptive meshing in approximate dynamic programming
title_short Volume-weighted Bellman error method for adaptive meshing in approximate dynamic programming
title_sort volume weighted bellman error method for adaptive meshing in approximate dynamic programming
topic control inteligente
programación dinámica aproximada
control óptimo
aprendizaje
url https://polipapers.upv.es/index.php/RIAI/article/view/15698
work_keys_str_mv AT leopoldoarmesto volumeweightedbellmanerrormethodforadaptivemeshinginapproximatedynamicprogramming
AT antoniosala volumeweightedbellmanerrormethodforadaptivemeshinginapproximatedynamicprogramming