Safe Nonlinear Control Under Control Constraints via Reachability, Optimal Control and Reinforcement Learning
Autonomous robots in the real world have nonlinear dynamics with actuators that are subject to constraints. The combination of the two poses complicates the task of designing stabilizing controllers that can guarantee safety, which we denote as the stabilize-avoid problem. Existing control-based tec...
| Main Author: | |
|---|---|
| Other Authors: | |
| Format: | Thesis |
| Published: |
Massachusetts Institute of Technology
2024
|
| Online Access: | https://hdl.handle.net/1721.1/155344 |
| _version_ | 1826214750934007808 |
|---|---|
| author | So, Oswin |
| author2 | Fan, Chuchu |
| author_facet | Fan, Chuchu So, Oswin |
| author_sort | So, Oswin |
| collection | MIT |
| description | Autonomous robots in the real world have nonlinear dynamics with actuators that are subject to constraints. The combination of the two poses complicates the task of designing stabilizing controllers that can guarantee safety, which we denote as the stabilize-avoid problem. Existing control-based techniques can provide safety and stability guarantees but under the assumption of unbounded control inputs. On the other hand, learning-based techniques can handle control constraints but often are unable to correctly trade-off between safety and stability.
In this thesis, we take a step towards synthesizing controllers with improved safety and stability for high dimensional nonlinear systems with control constraints by combining techniques from reachability, optimal control, and reinforcement learning. We first propose a novel approach to solve constrained optimal control problems using deep reinforcement learning by using techniques from traditional constrained optimization, enabling the solution of stabilize-avoid problems for high-dimensional nonlinear systems with control constraints. Next, we present an alternate method of solving the stabilize-avoid problem using control barrier functions,
where we present an improved method for learning control barrier functions for nonlinear systems with control constraints by drawing on connections between reachability and deep reinforcement learning.
We validate our proposed methods on a variety of benchmark tasks. Our experiments demonstrate the advantage of our methods over existing techniques in terms of improved safety rates and larger regions of attraction, especially in the case of high-dimensional systems. |
| first_indexed | 2024-09-23T16:10:35Z |
| format | Thesis |
| id | mit-1721.1/155344 |
| institution | Massachusetts Institute of Technology |
| last_indexed | 2024-09-23T16:10:35Z |
| publishDate | 2024 |
| publisher | Massachusetts Institute of Technology |
| record_format | dspace |
| spelling | mit-1721.1/1553442024-06-28T03:04:13Z Safe Nonlinear Control Under Control Constraints via Reachability, Optimal Control and Reinforcement Learning So, Oswin Fan, Chuchu Massachusetts Institute of Technology. Department of Aeronautics and Astronautics Autonomous robots in the real world have nonlinear dynamics with actuators that are subject to constraints. The combination of the two poses complicates the task of designing stabilizing controllers that can guarantee safety, which we denote as the stabilize-avoid problem. Existing control-based techniques can provide safety and stability guarantees but under the assumption of unbounded control inputs. On the other hand, learning-based techniques can handle control constraints but often are unable to correctly trade-off between safety and stability. In this thesis, we take a step towards synthesizing controllers with improved safety and stability for high dimensional nonlinear systems with control constraints by combining techniques from reachability, optimal control, and reinforcement learning. We first propose a novel approach to solve constrained optimal control problems using deep reinforcement learning by using techniques from traditional constrained optimization, enabling the solution of stabilize-avoid problems for high-dimensional nonlinear systems with control constraints. Next, we present an alternate method of solving the stabilize-avoid problem using control barrier functions, where we present an improved method for learning control barrier functions for nonlinear systems with control constraints by drawing on connections between reachability and deep reinforcement learning. We validate our proposed methods on a variety of benchmark tasks. Our experiments demonstrate the advantage of our methods over existing techniques in terms of improved safety rates and larger regions of attraction, especially in the case of high-dimensional systems. S.M. 2024-06-27T19:46:20Z 2024-06-27T19:46:20Z 2024-05 2024-05-28T19:36:36.012Z Thesis https://hdl.handle.net/1721.1/155344 0000-0002-5411-3663 In Copyright - Educational Use Permitted Copyright retained by author(s) https://rightsstatements.org/page/InC-EDU/1.0/ application/pdf Massachusetts Institute of Technology |
| spellingShingle | So, Oswin Safe Nonlinear Control Under Control Constraints via Reachability, Optimal Control and Reinforcement Learning |
| title | Safe Nonlinear Control Under Control Constraints via Reachability, Optimal Control and Reinforcement Learning |
| title_full | Safe Nonlinear Control Under Control Constraints via Reachability, Optimal Control and Reinforcement Learning |
| title_fullStr | Safe Nonlinear Control Under Control Constraints via Reachability, Optimal Control and Reinforcement Learning |
| title_full_unstemmed | Safe Nonlinear Control Under Control Constraints via Reachability, Optimal Control and Reinforcement Learning |
| title_short | Safe Nonlinear Control Under Control Constraints via Reachability, Optimal Control and Reinforcement Learning |
| title_sort | safe nonlinear control under control constraints via reachability optimal control and reinforcement learning |
| url | https://hdl.handle.net/1721.1/155344 |
| work_keys_str_mv | AT sooswin safenonlinearcontrolundercontrolconstraintsviareachabilityoptimalcontrolandreinforcementlearning |