Optimal adaptive control with separable drift uncertainty
We consider a problem of stochastic optimal control with separable drift uncertainty in strong formulation on a finite time horizon. The drift of the state Y u is multiplicatively influenced by an unknown random variable λ, while admissible controls u are required to be adapted to the observation fi...
| Main Authors: | , , |
|---|---|
| Format: | Journal article |
| Language: | English |
| Published: |
Society for Industrial and Applied Mathematics
2025
|
| _version_ | 1824458831034318848 |
|---|---|
| author | Cohen, SN Knochenhauer, C Merkel, A |
| author_facet | Cohen, SN Knochenhauer, C Merkel, A |
| author_sort | Cohen, SN |
| collection | OXFORD |
| description | We consider a problem of stochastic optimal control with separable drift uncertainty in strong formulation on a finite time horizon. The drift of the state Y u is multiplicatively influenced by an unknown random variable λ, while admissible controls u are required to be adapted to the observation filtration. Choosing a control actively influences the state and information acquisition simultaneously and comes with a learning effect. The problem, initially non-Markovian, is embedded into a higher-dimensional Markovian, full information control problem with control-dependent filtration and noise. To that problem, we apply the stochastic Perron method to characterize the value function as the unique viscosity solution of the HJB equation, explicitly construct ε-optimal controls, and show that the values in the strong and weak formulation agree. Numerical illustrations show a significant difference between the adaptive control and the certainty equivalence control, highlighting a substantial learning effect. |
| first_indexed | 2025-02-19T04:32:08Z |
| format | Journal article |
| id | oxford-uuid:ca85ed6f-2a3e-4e9b-89df-d77d135c15c7 |
| institution | University of Oxford |
| language | English |
| last_indexed | 2025-02-19T04:32:08Z |
| publishDate | 2025 |
| publisher | Society for Industrial and Applied Mathematics |
| record_format | dspace |
| spelling | oxford-uuid:ca85ed6f-2a3e-4e9b-89df-d77d135c15c72025-01-09T11:28:37ZOptimal adaptive control with separable drift uncertaintyJournal articlehttp://purl.org/coar/resource_type/c_dcae04bcuuid:ca85ed6f-2a3e-4e9b-89df-d77d135c15c7EnglishSymplectic ElementsSociety for Industrial and Applied Mathematics2025Cohen, SNKnochenhauer, CMerkel, AWe consider a problem of stochastic optimal control with separable drift uncertainty in strong formulation on a finite time horizon. The drift of the state Y u is multiplicatively influenced by an unknown random variable λ, while admissible controls u are required to be adapted to the observation filtration. Choosing a control actively influences the state and information acquisition simultaneously and comes with a learning effect. The problem, initially non-Markovian, is embedded into a higher-dimensional Markovian, full information control problem with control-dependent filtration and noise. To that problem, we apply the stochastic Perron method to characterize the value function as the unique viscosity solution of the HJB equation, explicitly construct ε-optimal controls, and show that the values in the strong and weak formulation agree. Numerical illustrations show a significant difference between the adaptive control and the certainty equivalence control, highlighting a substantial learning effect. |
| spellingShingle | Cohen, SN Knochenhauer, C Merkel, A Optimal adaptive control with separable drift uncertainty |
| title | Optimal adaptive control with separable drift uncertainty |
| title_full | Optimal adaptive control with separable drift uncertainty |
| title_fullStr | Optimal adaptive control with separable drift uncertainty |
| title_full_unstemmed | Optimal adaptive control with separable drift uncertainty |
| title_short | Optimal adaptive control with separable drift uncertainty |
| title_sort | optimal adaptive control with separable drift uncertainty |
| work_keys_str_mv | AT cohensn optimaladaptivecontrolwithseparabledriftuncertainty AT knochenhauerc optimaladaptivecontrolwithseparabledriftuncertainty AT merkela optimaladaptivecontrolwithseparabledriftuncertainty |