Gradient bounded dynamic programming with submodular and concave extensible value functions
We consider dynamic programming problems with finite, discrete-time horizons and prohibitively high-dimensional, discrete state-spaces for direct computation of the value function from the Bellman equation. For the case that the value function of the dynamic program is concave extensible and submodu...
| Main Authors: | , , |
|---|---|
| Format: | Conference item |
| Language: | English |
| Published: |
Elsevier
2021
|
| _version_ | 1826296294845448192 |
|---|---|
| author | Lebedev, D Goulart, P Margellos, K |
| author_facet | Lebedev, D Goulart, P Margellos, K |
| author_sort | Lebedev, D |
| collection | OXFORD |
| description | We consider dynamic programming problems with finite, discrete-time horizons and prohibitively high-dimensional, discrete state-spaces for direct computation of the value function from the Bellman equation. For the case that the value function of the dynamic program is concave extensible and submodular in its state-space, we present a new algorithm that computes deterministic upper and stochastic lower bounds of the value function similar to dual dynamic programming. We then show that the proposed algorithm terminates after a fnite number of iterations. Finally, we demonstrate the efficacy of our approach on a high-dimensional numerical example from delivery slot pricing in attended home delivery. |
| first_indexed | 2024-03-07T04:14:08Z |
| format | Conference item |
| id | oxford-uuid:c8d6b6bd-8c1c-4782-bd4f-b82ed1c39a6c |
| institution | University of Oxford |
| language | English |
| last_indexed | 2024-03-07T04:14:08Z |
| publishDate | 2021 |
| publisher | Elsevier |
| record_format | dspace |
| spelling | oxford-uuid:c8d6b6bd-8c1c-4782-bd4f-b82ed1c39a6c2022-03-27T06:54:55ZGradient bounded dynamic programming with submodular and concave extensible value functionsConference itemhttp://purl.org/coar/resource_type/c_5794uuid:c8d6b6bd-8c1c-4782-bd4f-b82ed1c39a6cEnglishSymplectic ElementsElsevier2021Lebedev, DGoulart, PMargellos, KWe consider dynamic programming problems with finite, discrete-time horizons and prohibitively high-dimensional, discrete state-spaces for direct computation of the value function from the Bellman equation. For the case that the value function of the dynamic program is concave extensible and submodular in its state-space, we present a new algorithm that computes deterministic upper and stochastic lower bounds of the value function similar to dual dynamic programming. We then show that the proposed algorithm terminates after a fnite number of iterations. Finally, we demonstrate the efficacy of our approach on a high-dimensional numerical example from delivery slot pricing in attended home delivery. |
| spellingShingle | Lebedev, D Goulart, P Margellos, K Gradient bounded dynamic programming with submodular and concave extensible value functions |
| title | Gradient bounded dynamic programming with submodular and concave extensible value functions |
| title_full | Gradient bounded dynamic programming with submodular and concave extensible value functions |
| title_fullStr | Gradient bounded dynamic programming with submodular and concave extensible value functions |
| title_full_unstemmed | Gradient bounded dynamic programming with submodular and concave extensible value functions |
| title_short | Gradient bounded dynamic programming with submodular and concave extensible value functions |
| title_sort | gradient bounded dynamic programming with submodular and concave extensible value functions |
| work_keys_str_mv | AT lebedevd gradientboundeddynamicprogrammingwithsubmodularandconcaveextensiblevaluefunctions AT goulartp gradientboundeddynamicprogrammingwithsubmodularandconcaveextensiblevaluefunctions AT margellosk gradientboundeddynamicprogrammingwithsubmodularandconcaveextensiblevaluefunctions |