Deep Reinforcement Learning Tf-Agent-Based Object Tracking With Virtual Autonomous Drone in a Game Engine
The recent development of object-tracking frameworks has affected the performance of many manufacturing and industrial services such as product delivery, autonomous driving systems, security systems, military, transportation and retailing industries, smart cities, healthcare systems, agriculture, et...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2023-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10286478/ |
| _version_ | 1827763178229989376 |
|---|---|
| author | Khurshedjon Farkhodov Suk-Hwan Lee Jan Platos Ki-Ryong Kwon |
| author_facet | Khurshedjon Farkhodov Suk-Hwan Lee Jan Platos Ki-Ryong Kwon |
| author_sort | Khurshedjon Farkhodov |
| collection | DOAJ |
| description | The recent development of object-tracking frameworks has affected the performance of many manufacturing and industrial services such as product delivery, autonomous driving systems, security systems, military, transportation and retailing industries, smart cities, healthcare systems, agriculture, etc. Achieving accurate results in physical environments and conditions remains quite challenging for the actual object-tracking. However, the process can be experimented with using simulation techniques or platforms to evaluate and check the model’s performance under different simulation conditions and weather changes. This paper presents one of the target tracking approaches based on the reinforcement learning technique integrated with TensorFlow-Agent (tf-agent) to accomplish the tracking process in the Unreal Game Engine simulation platform AirSim Blocks. The productivity of these platforms can be seen while experimenting in virtual-reality conditions with virtual drone agents and performing fine-tuning to achieve the best or desired performance. In this paper, the tf-agent drone learns how to track an object integration with a deep reinforcement learning process to control the actions, states, and tracking by receiving sequential frames from a simple Blocks environment. The tf-agent model is trained in the AirSim Blocks environment for adaptation to the environment and existing objects in a simulation environment for further testing and evaluation regarding the accuracy of tracking and speed. We tested and compared two approaches, DQN and PPO trackers, and reported results in terms of stability, rewards, and numerical performance. |
| first_indexed | 2024-03-11T10:48:29Z |
| format | Article |
| id | doaj.art-e1b2b1a047d943c4a28534939e794c3e |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-03-11T10:48:29Z |
| publishDate | 2023-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-e1b2b1a047d943c4a28534939e794c3e2023-11-14T00:00:49ZengIEEEIEEE Access2169-35362023-01-011112412912413810.1109/ACCESS.2023.332506210286478Deep Reinforcement Learning Tf-Agent-Based Object Tracking With Virtual Autonomous Drone in a Game EngineKhurshedjon Farkhodov0https://orcid.org/0000-0003-0912-8767Suk-Hwan Lee1Jan Platos2https://orcid.org/0000-0002-8481-0136Ki-Ryong Kwon3https://orcid.org/0000-0002-1879-748XDepartment of AI Convergence, Pukyong National University, Busan, South KoreaDepartment of Computer Engineering, Dong-A University, Busan, South KoreaDepartment of Electrical Engineering and Computer Science, VSB—Technical University of Ostrava, Ostrava, Czech RepublicDepartment of AI Convergence, Pukyong National University, Busan, South KoreaThe recent development of object-tracking frameworks has affected the performance of many manufacturing and industrial services such as product delivery, autonomous driving systems, security systems, military, transportation and retailing industries, smart cities, healthcare systems, agriculture, etc. Achieving accurate results in physical environments and conditions remains quite challenging for the actual object-tracking. However, the process can be experimented with using simulation techniques or platforms to evaluate and check the model’s performance under different simulation conditions and weather changes. This paper presents one of the target tracking approaches based on the reinforcement learning technique integrated with TensorFlow-Agent (tf-agent) to accomplish the tracking process in the Unreal Game Engine simulation platform AirSim Blocks. The productivity of these platforms can be seen while experimenting in virtual-reality conditions with virtual drone agents and performing fine-tuning to achieve the best or desired performance. In this paper, the tf-agent drone learns how to track an object integration with a deep reinforcement learning process to control the actions, states, and tracking by receiving sequential frames from a simple Blocks environment. The tf-agent model is trained in the AirSim Blocks environment for adaptation to the environment and existing objects in a simulation environment for further testing and evaluation regarding the accuracy of tracking and speed. We tested and compared two approaches, DQN and PPO trackers, and reported results in terms of stability, rewards, and numerical performance.https://ieeexplore.ieee.org/document/10286478/Object trackingobject detectionreinforcement learningAirSimvirtual environmentvirtual simulation |
| spellingShingle | Khurshedjon Farkhodov Suk-Hwan Lee Jan Platos Ki-Ryong Kwon Deep Reinforcement Learning Tf-Agent-Based Object Tracking With Virtual Autonomous Drone in a Game Engine IEEE Access Object tracking object detection reinforcement learning AirSim virtual environment virtual simulation |
| title | Deep Reinforcement Learning Tf-Agent-Based Object Tracking With Virtual Autonomous Drone in a Game Engine |
| title_full | Deep Reinforcement Learning Tf-Agent-Based Object Tracking With Virtual Autonomous Drone in a Game Engine |
| title_fullStr | Deep Reinforcement Learning Tf-Agent-Based Object Tracking With Virtual Autonomous Drone in a Game Engine |
| title_full_unstemmed | Deep Reinforcement Learning Tf-Agent-Based Object Tracking With Virtual Autonomous Drone in a Game Engine |
| title_short | Deep Reinforcement Learning Tf-Agent-Based Object Tracking With Virtual Autonomous Drone in a Game Engine |
| title_sort | deep reinforcement learning tf agent based object tracking with virtual autonomous drone in a game engine |
| topic | Object tracking object detection reinforcement learning AirSim virtual environment virtual simulation |
| url | https://ieeexplore.ieee.org/document/10286478/ |
| work_keys_str_mv | AT khurshedjonfarkhodov deepreinforcementlearningtfagentbasedobjecttrackingwithvirtualautonomousdroneinagameengine AT sukhwanlee deepreinforcementlearningtfagentbasedobjecttrackingwithvirtualautonomousdroneinagameengine AT janplatos deepreinforcementlearningtfagentbasedobjecttrackingwithvirtualautonomousdroneinagameengine AT kiryongkwon deepreinforcementlearningtfagentbasedobjecttrackingwithvirtualautonomousdroneinagameengine |