Actor–Critic Reinforcement Learning and Application in Developing Computer-Vision-Based Interface Tracking
This paper synchronizes control theory with computer vision by formalizing object tracking as a sequential decision-making process. A reinforcement learning (RL) agent successfully tracks an interface between two liquids, which is often a critical variable to track in many chemical, petrochemical, m...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Elsevier
2021-09-01
|
| Series: | Engineering |
| Subjects: | |
| Online Access: | http://www.sciencedirect.com/science/article/pii/S209580992100326X |
| _version_ | 1798033252560142336 |
|---|---|
| author | Oguzhan Dogru Kirubakaran Velswamy Biao Huang |
| author_facet | Oguzhan Dogru Kirubakaran Velswamy Biao Huang |
| author_sort | Oguzhan Dogru |
| collection | DOAJ |
| description | This paper synchronizes control theory with computer vision by formalizing object tracking as a sequential decision-making process. A reinforcement learning (RL) agent successfully tracks an interface between two liquids, which is often a critical variable to track in many chemical, petrochemical, metallurgical, and oil industries. This method utilizes less than 100 images for creating an environment, from which the agent generates its own data without the need for expert knowledge. Unlike supervised learning (SL) methods that rely on a huge number of parameters, this approach requires far fewer parameters, which naturally reduces its maintenance cost. Besides its frugal nature, the agent is robust to environmental uncertainties such as occlusion, intensity changes, and excessive noise. From a closed-loop control context, an interface location-based deviation is chosen as the optimization goal during training. The methodology showcases RL for real-time object-tracking applications in the oil sands industry. Along with a presentation of the interface tracking problem, this paper provides a detailed review of one of the most effective RL methodologies: actor–critic policy. |
| first_indexed | 2024-04-11T20:27:24Z |
| format | Article |
| id | doaj.art-a624e2d1afb747039710f977d50ab005 |
| institution | Directory Open Access Journal |
| issn | 2095-8099 |
| language | English |
| last_indexed | 2024-04-11T20:27:24Z |
| publishDate | 2021-09-01 |
| publisher | Elsevier |
| record_format | Article |
| series | Engineering |
| spelling | doaj.art-a624e2d1afb747039710f977d50ab0052022-12-22T04:04:38ZengElsevierEngineering2095-80992021-09-017912481261Actor–Critic Reinforcement Learning and Application in Developing Computer-Vision-Based Interface TrackingOguzhan Dogru0Kirubakaran Velswamy1Biao Huang2Department of Chemical and Materials Engineering, University of Alberta, Edmonton, AB T6G 1H9, CanadaDepartment of Chemical and Materials Engineering, University of Alberta, Edmonton, AB T6G 1H9, CanadaCorresponding author.; Department of Chemical and Materials Engineering, University of Alberta, Edmonton, AB T6G 1H9, CanadaThis paper synchronizes control theory with computer vision by formalizing object tracking as a sequential decision-making process. A reinforcement learning (RL) agent successfully tracks an interface between two liquids, which is often a critical variable to track in many chemical, petrochemical, metallurgical, and oil industries. This method utilizes less than 100 images for creating an environment, from which the agent generates its own data without the need for expert knowledge. Unlike supervised learning (SL) methods that rely on a huge number of parameters, this approach requires far fewer parameters, which naturally reduces its maintenance cost. Besides its frugal nature, the agent is robust to environmental uncertainties such as occlusion, intensity changes, and excessive noise. From a closed-loop control context, an interface location-based deviation is chosen as the optimization goal during training. The methodology showcases RL for real-time object-tracking applications in the oil sands industry. Along with a presentation of the interface tracking problem, this paper provides a detailed review of one of the most effective RL methodologies: actor–critic policy.http://www.sciencedirect.com/science/article/pii/S209580992100326XInterface trackingObject trackingOcclusionReinforcement learningUniform manifold approximation and projection |
| spellingShingle | Oguzhan Dogru Kirubakaran Velswamy Biao Huang Actor–Critic Reinforcement Learning and Application in Developing Computer-Vision-Based Interface Tracking Engineering Interface tracking Object tracking Occlusion Reinforcement learning Uniform manifold approximation and projection |
| title | Actor–Critic Reinforcement Learning and Application in Developing Computer-Vision-Based Interface Tracking |
| title_full | Actor–Critic Reinforcement Learning and Application in Developing Computer-Vision-Based Interface Tracking |
| title_fullStr | Actor–Critic Reinforcement Learning and Application in Developing Computer-Vision-Based Interface Tracking |
| title_full_unstemmed | Actor–Critic Reinforcement Learning and Application in Developing Computer-Vision-Based Interface Tracking |
| title_short | Actor–Critic Reinforcement Learning and Application in Developing Computer-Vision-Based Interface Tracking |
| title_sort | actor critic reinforcement learning and application in developing computer vision based interface tracking |
| topic | Interface tracking Object tracking Occlusion Reinforcement learning Uniform manifold approximation and projection |
| url | http://www.sciencedirect.com/science/article/pii/S209580992100326X |
| work_keys_str_mv | AT oguzhandogru actorcriticreinforcementlearningandapplicationindevelopingcomputervisionbasedinterfacetracking AT kirubakaranvelswamy actorcriticreinforcementlearningandapplicationindevelopingcomputervisionbasedinterfacetracking AT biaohuang actorcriticreinforcementlearningandapplicationindevelopingcomputervisionbasedinterfacetracking |