Active perception in adversarial scenarios using maximum entropy deep reinforcement learning
© 2019 IEEE. We pose an active perception problem where an autonomous agent actively interacts with a second agent with potentially adversarial behaviors. Given the uncertainty in the intent of the other agent, the objective is to collect further evidence to help discriminate potential threats. The...
| Main Authors: | , |
|---|---|
| Other Authors: | |
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2021
|
| Online Access: | https://hdl.handle.net/1721.1/137865.2 |
| _version_ | 1811097650033852416 |
|---|---|
| author | Shen, Macheng How, Jonathan P. |
| author2 | Massachusetts Institute of Technology. Department of Aeronautics and Astronautics |
| author_facet | Massachusetts Institute of Technology. Department of Aeronautics and Astronautics Shen, Macheng How, Jonathan P. |
| author_sort | Shen, Macheng |
| collection | MIT |
| description | © 2019 IEEE. We pose an active perception problem where an autonomous agent actively interacts with a second agent with potentially adversarial behaviors. Given the uncertainty in the intent of the other agent, the objective is to collect further evidence to help discriminate potential threats. The main technical challenges are the partial observability of the agent intent, the adversary modeling, and the corresponding uncertainty modeling. Note that an adversary agent may act to mislead the autonomous agent by using a deceptive strategy that is learned from past experiences. We propose an approach that combines belief space planning, generative adversary modeling, and maximum entropy reinforcement learning to obtain a stochastic belief space policy. By accounting for various adversarial behaviors in the simulation framework and minimizing the predictability of the autonomous agent's action, the resulting policy is more robust to unmodeled adversarial strategies. This improved robustness is empirically shown against an adversary that adapts to and exploits the autonomous agent's policy when compared with a standard Chance-Constraint Partially Observable Markov Decision Process robust approach. |
| first_indexed | 2024-09-23T17:02:48Z |
| format | Article |
| id | mit-1721.1/137865.2 |
| institution | Massachusetts Institute of Technology |
| language | English |
| last_indexed | 2024-09-23T17:02:48Z |
| publishDate | 2021 |
| publisher | IEEE |
| record_format | dspace |
| spelling | mit-1721.1/137865.22021-11-09T21:14:39Z Active perception in adversarial scenarios using maximum entropy deep reinforcement learning Shen, Macheng How, Jonathan P. Massachusetts Institute of Technology. Department of Aeronautics and Astronautics Massachusetts Institute of Technology. Department of Mechanical Engineering © 2019 IEEE. We pose an active perception problem where an autonomous agent actively interacts with a second agent with potentially adversarial behaviors. Given the uncertainty in the intent of the other agent, the objective is to collect further evidence to help discriminate potential threats. The main technical challenges are the partial observability of the agent intent, the adversary modeling, and the corresponding uncertainty modeling. Note that an adversary agent may act to mislead the autonomous agent by using a deceptive strategy that is learned from past experiences. We propose an approach that combines belief space planning, generative adversary modeling, and maximum entropy reinforcement learning to obtain a stochastic belief space policy. By accounting for various adversarial behaviors in the simulation framework and minimizing the predictability of the autonomous agent's action, the resulting policy is more robust to unmodeled adversarial strategies. This improved robustness is empirically shown against an adversary that adapts to and exploits the autonomous agent's policy when compared with a standard Chance-Constraint Partially Observable Markov Decision Process robust approach. ARL (Award W911NF-17-2-0181) 2021-11-09T21:14:38Z 2021-11-09T13:48:00Z 2021-11-09T21:14:38Z 2019-09 2019-10-28T17:33:35Z Article http://purl.org/eprint/type/ConferencePaper https://hdl.handle.net/1721.1/137865.2 Shen, Macheng and How, Jonathan P. 2019. "Active perception in adversarial scenarios using maximum entropy deep reinforcement learning." en 10.1109/ICRA.2019.8794389 Creative Commons Attribution-Noncommercial-Share Alike http://creativecommons.org/licenses/by-nc-sa/4.0/ application/octet-stream IEEE arXiv |
| spellingShingle | Shen, Macheng How, Jonathan P. Active perception in adversarial scenarios using maximum entropy deep reinforcement learning |
| title | Active perception in adversarial scenarios using maximum entropy deep reinforcement learning |
| title_full | Active perception in adversarial scenarios using maximum entropy deep reinforcement learning |
| title_fullStr | Active perception in adversarial scenarios using maximum entropy deep reinforcement learning |
| title_full_unstemmed | Active perception in adversarial scenarios using maximum entropy deep reinforcement learning |
| title_short | Active perception in adversarial scenarios using maximum entropy deep reinforcement learning |
| title_sort | active perception in adversarial scenarios using maximum entropy deep reinforcement learning |
| url | https://hdl.handle.net/1721.1/137865.2 |
| work_keys_str_mv | AT shenmacheng activeperceptioninadversarialscenariosusingmaximumentropydeepreinforcementlearning AT howjonathanp activeperceptioninadversarialscenariosusingmaximumentropydeepreinforcementlearning |