Learning to Rapidly Re-Contact the Lost Plume in Chemical Plume Tracing
Maintaining contact between the robot and plume is significant in chemical plume tracing (CPT). In the time immediately following the loss of chemical detection during the process of CPT, Track-Out activities bias the robot heading relative to the upwind direction, expecting to rapidly re-contact th...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2015-03-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | http://www.mdpi.com/1424-8220/15/4/7512 |
| _version_ | 1811277874369396736 |
|---|---|
| author | Meng-Li Cao Qing-Hao Meng Jia-Ying Wang Bing Luo Ya-Qi Jing Shu-Gen Ma |
| author_facet | Meng-Li Cao Qing-Hao Meng Jia-Ying Wang Bing Luo Ya-Qi Jing Shu-Gen Ma |
| author_sort | Meng-Li Cao |
| collection | DOAJ |
| description | Maintaining contact between the robot and plume is significant in chemical plume tracing (CPT). In the time immediately following the loss of chemical detection during the process of CPT, Track-Out activities bias the robot heading relative to the upwind direction, expecting to rapidly re-contact the plume. To determine the bias angle used in the Track-Out activity, we propose an online instance-based reinforcement learning method, namely virtual trail following (VTF). In VTF, action-value is generalized from recently stored instances of successful Track-Out activities. We also propose a collaborative VTF (cVTF) method, in which multiple robots store their own instances, and learn from the stored instances, in the same database. The proposed VTF and cVTF methods are compared with biased upwind surge (BUS) method, in which all Track-Out activities utilize an offline optimized universal bias angle, in an indoor environment with three different airflow fields. With respect to our experimental conditions, VTF and cVTF show stronger adaptability to different airflow environments than BUS, and furthermore, cVTF yields higher success rates and time-efficiencies than VTF. |
| first_indexed | 2024-04-13T00:24:25Z |
| format | Article |
| id | doaj.art-68f56c141bd3489e85569ae0041a317c |
| institution | Directory Open Access Journal |
| issn | 1424-8220 |
| language | English |
| last_indexed | 2024-04-13T00:24:25Z |
| publishDate | 2015-03-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj.art-68f56c141bd3489e85569ae0041a317c2022-12-22T03:10:38ZengMDPI AGSensors1424-82202015-03-011547512753610.3390/s150407512s150407512Learning to Rapidly Re-Contact the Lost Plume in Chemical Plume TracingMeng-Li Cao0Qing-Hao Meng1Jia-Ying Wang2Bing Luo3Ya-Qi Jing4Shu-Gen Ma5Institute of Robotics and Autonomous Systems, Tianjin Key Laboratory of Process Measurement and Control, School of Electrical Engineering and Automation, Tianjin University, Tianjin 300072, ChinaInstitute of Robotics and Autonomous Systems, Tianjin Key Laboratory of Process Measurement and Control, School of Electrical Engineering and Automation, Tianjin University, Tianjin 300072, ChinaInstitute of Robotics and Autonomous Systems, Tianjin Key Laboratory of Process Measurement and Control, School of Electrical Engineering and Automation, Tianjin University, Tianjin 300072, ChinaInstitute of Robotics and Autonomous Systems, Tianjin Key Laboratory of Process Measurement and Control, School of Electrical Engineering and Automation, Tianjin University, Tianjin 300072, ChinaInstitute of Robotics and Autonomous Systems, Tianjin Key Laboratory of Process Measurement and Control, School of Electrical Engineering and Automation, Tianjin University, Tianjin 300072, ChinaInstitute of Robotics and Autonomous Systems, Tianjin Key Laboratory of Process Measurement and Control, School of Electrical Engineering and Automation, Tianjin University, Tianjin 300072, ChinaMaintaining contact between the robot and plume is significant in chemical plume tracing (CPT). In the time immediately following the loss of chemical detection during the process of CPT, Track-Out activities bias the robot heading relative to the upwind direction, expecting to rapidly re-contact the plume. To determine the bias angle used in the Track-Out activity, we propose an online instance-based reinforcement learning method, namely virtual trail following (VTF). In VTF, action-value is generalized from recently stored instances of successful Track-Out activities. We also propose a collaborative VTF (cVTF) method, in which multiple robots store their own instances, and learn from the stored instances, in the same database. The proposed VTF and cVTF methods are compared with biased upwind surge (BUS) method, in which all Track-Out activities utilize an offline optimized universal bias angle, in an indoor environment with three different airflow fields. With respect to our experimental conditions, VTF and cVTF show stronger adaptability to different airflow environments than BUS, and furthermore, cVTF yields higher success rates and time-efficiencies than VTF.http://www.mdpi.com/1424-8220/15/4/7512chemical plume tracingreinforcement learningcollaborative learningbehavior-based robotics |
| spellingShingle | Meng-Li Cao Qing-Hao Meng Jia-Ying Wang Bing Luo Ya-Qi Jing Shu-Gen Ma Learning to Rapidly Re-Contact the Lost Plume in Chemical Plume Tracing Sensors chemical plume tracing reinforcement learning collaborative learning behavior-based robotics |
| title | Learning to Rapidly Re-Contact the Lost Plume in Chemical Plume Tracing |
| title_full | Learning to Rapidly Re-Contact the Lost Plume in Chemical Plume Tracing |
| title_fullStr | Learning to Rapidly Re-Contact the Lost Plume in Chemical Plume Tracing |
| title_full_unstemmed | Learning to Rapidly Re-Contact the Lost Plume in Chemical Plume Tracing |
| title_short | Learning to Rapidly Re-Contact the Lost Plume in Chemical Plume Tracing |
| title_sort | learning to rapidly re contact the lost plume in chemical plume tracing |
| topic | chemical plume tracing reinforcement learning collaborative learning behavior-based robotics |
| url | http://www.mdpi.com/1424-8220/15/4/7512 |
| work_keys_str_mv | AT menglicao learningtorapidlyrecontactthelostplumeinchemicalplumetracing AT qinghaomeng learningtorapidlyrecontactthelostplumeinchemicalplumetracing AT jiayingwang learningtorapidlyrecontactthelostplumeinchemicalplumetracing AT bingluo learningtorapidlyrecontactthelostplumeinchemicalplumetracing AT yaqijing learningtorapidlyrecontactthelostplumeinchemicalplumetracing AT shugenma learningtorapidlyrecontactthelostplumeinchemicalplumetracing |