Deep Reinforcement Learning Object Tracking Based on Actor-Double Critic Network
Aiming at the problem of poor tracking robustness caused by severe occlusion, deformation, and object rotation of deep learning object tracking algorithm in complex scenes, an improved deep reinforcement learning object tracking algorithm based on actor-double critic network is proposed. In offline...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Tsinghua University Press
2023-12-01
|
| Series: | CAAI Artificial Intelligence Research |
| Subjects: | |
| Online Access: | https://www.sciopen.com/article/10.26599/AIR.2023.9150013 |
| _version_ | 1797202593306378240 |
|---|---|
| author | Jing Xin Jianglei Zhou Xinhong Hei Pengyu Yue Jia Zhao |
| author_facet | Jing Xin Jianglei Zhou Xinhong Hei Pengyu Yue Jia Zhao |
| author_sort | Jing Xin |
| collection | DOAJ |
| description | Aiming at the problem of poor tracking robustness caused by severe occlusion, deformation, and object rotation of deep learning object tracking algorithm in complex scenes, an improved deep reinforcement learning object tracking algorithm based on actor-double critic network is proposed. In offline training phase, the actor network moves the rectangular box representing the object location according to the input sequence image to obtain the action value, that is, the horizontal, vertical, and scale transformation of the object. Then, the designed double critic network is used to evaluate the action value, and the output double Q value is averaged to guide the actor network to optimize the tracking strategy. The design of double critic network effectively improves the stability and convergence, especially in challenging scenes such as object occlusion, and the tracking performance is significantly improved. In online tracking phase, the well-trained actor network is used to infer the changing action of the bounding box, directly causing the tracker to move the box to the object position in the current frame. Several comparative tracking experiments were conducted on the OTB100 visual tracker benchmark and the experimental results show that more intensive reward settings significantly increase the actor network’s output probability of positive actions. This makes the tracking algorithm proposed in this paper outperforms the mainstream deep reinforcement learning tracking algorithms and deep learning tracking algorithms under the challenging attributes such as occlusion, deformation, and rotation. |
| first_indexed | 2024-04-24T08:05:54Z |
| format | Article |
| id | doaj.art-455b8c9ecbd34549833127a71f3cdde7 |
| institution | Directory Open Access Journal |
| issn | 2097-194X |
| language | English |
| last_indexed | 2024-04-24T08:05:54Z |
| publishDate | 2023-12-01 |
| publisher | Tsinghua University Press |
| record_format | Article |
| series | CAAI Artificial Intelligence Research |
| spelling | doaj.art-455b8c9ecbd34549833127a71f3cdde72024-04-17T10:29:52ZengTsinghua University PressCAAI Artificial Intelligence Research2097-194X2023-12-012915001310.26599/AIR.2023.9150013Deep Reinforcement Learning Object Tracking Based on Actor-Double Critic NetworkJing Xin0Jianglei Zhou1Xinhong Hei2Pengyu Yue3Jia Zhao4School of Automation and Information Engineering, Xi’an University of Technology, Xi’an 710048, ChinaSchool of Automation and Information Engineering, Xi’an University of Technology, Xi’an 710048, ChinaSchool of Computer Science and Engineering, Xi’an University of Technology, Xi’an 710048, ChinaSchool of Automation and Information Engineering, Xi’an University of Technology, Xi’an 710048, ChinaSchool of Automation and Information Engineering, Xi’an University of Technology, Xi’an 710048, ChinaAiming at the problem of poor tracking robustness caused by severe occlusion, deformation, and object rotation of deep learning object tracking algorithm in complex scenes, an improved deep reinforcement learning object tracking algorithm based on actor-double critic network is proposed. In offline training phase, the actor network moves the rectangular box representing the object location according to the input sequence image to obtain the action value, that is, the horizontal, vertical, and scale transformation of the object. Then, the designed double critic network is used to evaluate the action value, and the output double Q value is averaged to guide the actor network to optimize the tracking strategy. The design of double critic network effectively improves the stability and convergence, especially in challenging scenes such as object occlusion, and the tracking performance is significantly improved. In online tracking phase, the well-trained actor network is used to infer the changing action of the bounding box, directly causing the tracker to move the box to the object position in the current frame. Several comparative tracking experiments were conducted on the OTB100 visual tracker benchmark and the experimental results show that more intensive reward settings significantly increase the actor network’s output probability of positive actions. This makes the tracking algorithm proposed in this paper outperforms the mainstream deep reinforcement learning tracking algorithms and deep learning tracking algorithms under the challenging attributes such as occlusion, deformation, and rotation.https://www.sciopen.com/article/10.26599/AIR.2023.9150013object trackingdeep reinforcement learningactor-double critic network |
| spellingShingle | Jing Xin Jianglei Zhou Xinhong Hei Pengyu Yue Jia Zhao Deep Reinforcement Learning Object Tracking Based on Actor-Double Critic Network CAAI Artificial Intelligence Research object tracking deep reinforcement learning actor-double critic network |
| title | Deep Reinforcement Learning Object Tracking Based on Actor-Double Critic Network |
| title_full | Deep Reinforcement Learning Object Tracking Based on Actor-Double Critic Network |
| title_fullStr | Deep Reinforcement Learning Object Tracking Based on Actor-Double Critic Network |
| title_full_unstemmed | Deep Reinforcement Learning Object Tracking Based on Actor-Double Critic Network |
| title_short | Deep Reinforcement Learning Object Tracking Based on Actor-Double Critic Network |
| title_sort | deep reinforcement learning object tracking based on actor double critic network |
| topic | object tracking deep reinforcement learning actor-double critic network |
| url | https://www.sciopen.com/article/10.26599/AIR.2023.9150013 |
| work_keys_str_mv | AT jingxin deepreinforcementlearningobjecttrackingbasedonactordoublecriticnetwork AT jiangleizhou deepreinforcementlearningobjecttrackingbasedonactordoublecriticnetwork AT xinhonghei deepreinforcementlearningobjecttrackingbasedonactordoublecriticnetwork AT pengyuyue deepreinforcementlearningobjecttrackingbasedonactordoublecriticnetwork AT jiazhao deepreinforcementlearningobjecttrackingbasedonactordoublecriticnetwork |