C2RL: Convolutional-Contrastive Learning for Reinforcement Learning Based on Self-Pretraining for Strong Augmentation
Reinforcement learning agents that have not been seen during training must be robust in test environments. However, the generalization problem is challenging to solve in reinforcement learning using high-dimensional images as the input. The addition of a self-supervised learning framework with data...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-05-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/23/10/4946 |
| _version_ | 1797598326261022720 |
|---|---|
| author | Sanghoon Park Jihun Kim Han-You Jeong Tae-Kyoung Kim Jinwoo Yoo |
| author_facet | Sanghoon Park Jihun Kim Han-You Jeong Tae-Kyoung Kim Jinwoo Yoo |
| author_sort | Sanghoon Park |
| collection | DOAJ |
| description | Reinforcement learning agents that have not been seen during training must be robust in test environments. However, the generalization problem is challenging to solve in reinforcement learning using high-dimensional images as the input. The addition of a self-supervised learning framework with data augmentation in the reinforcement learning architecture can promote generalization to a certain extent. However, excessively large changes in the input images may disturb reinforcement learning. Therefore, we propose a contrastive learning method that can help manage the trade-off relationship between the performance of reinforcement learning and auxiliary tasks against the data augmentation strength. In this framework, strong augmentation does not disturb reinforcement learning and instead maximizes the auxiliary effect for generalization. Results of experiments on the DeepMind Control suite demonstrate that the proposed method effectively uses strong data augmentation and achieves a higher generalization than the existing methods. |
| first_indexed | 2024-03-11T03:20:40Z |
| format | Article |
| id | doaj.art-adfe85933d764462b338ac3f09a313b5 |
| institution | Directory Open Access Journal |
| issn | 1424-8220 |
| language | English |
| last_indexed | 2024-03-11T03:20:40Z |
| publishDate | 2023-05-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj.art-adfe85933d764462b338ac3f09a313b52023-11-18T03:14:59ZengMDPI AGSensors1424-82202023-05-012310494610.3390/s23104946C2RL: Convolutional-Contrastive Learning for Reinforcement Learning Based on Self-Pretraining for Strong AugmentationSanghoon Park0Jihun Kim1Han-You Jeong2Tae-Kyoung Kim3Jinwoo Yoo4Graduate School of Automotive Engineering, Kookmin University, Seoul 02707, Republic of KoreaGraduate School of Automotive Engineering, Kookmin University, Seoul 02707, Republic of KoreaDepartment of Electrical Engineering, Pusan National University, Busan 46241, Republic of KoreaDepartment of Electronic Engineering, Gachon University, Seongnam 13120, Republic of KoreaDepartment of Automobile and IT Convergence, Kookmin University, Seoul 02707, Republic of KoreaReinforcement learning agents that have not been seen during training must be robust in test environments. However, the generalization problem is challenging to solve in reinforcement learning using high-dimensional images as the input. The addition of a self-supervised learning framework with data augmentation in the reinforcement learning architecture can promote generalization to a certain extent. However, excessively large changes in the input images may disturb reinforcement learning. Therefore, we propose a contrastive learning method that can help manage the trade-off relationship between the performance of reinforcement learning and auxiliary tasks against the data augmentation strength. In this framework, strong augmentation does not disturb reinforcement learning and instead maximizes the auxiliary effect for generalization. Results of experiments on the DeepMind Control suite demonstrate that the proposed method effectively uses strong data augmentation and achieves a higher generalization than the existing methods.https://www.mdpi.com/1424-8220/23/10/4946deep reinforcement learningself-supervised learningcontrastive learninggeneralizationdata augmentationnetwork randomization |
| spellingShingle | Sanghoon Park Jihun Kim Han-You Jeong Tae-Kyoung Kim Jinwoo Yoo C2RL: Convolutional-Contrastive Learning for Reinforcement Learning Based on Self-Pretraining for Strong Augmentation Sensors deep reinforcement learning self-supervised learning contrastive learning generalization data augmentation network randomization |
| title | C2RL: Convolutional-Contrastive Learning for Reinforcement Learning Based on Self-Pretraining for Strong Augmentation |
| title_full | C2RL: Convolutional-Contrastive Learning for Reinforcement Learning Based on Self-Pretraining for Strong Augmentation |
| title_fullStr | C2RL: Convolutional-Contrastive Learning for Reinforcement Learning Based on Self-Pretraining for Strong Augmentation |
| title_full_unstemmed | C2RL: Convolutional-Contrastive Learning for Reinforcement Learning Based on Self-Pretraining for Strong Augmentation |
| title_short | C2RL: Convolutional-Contrastive Learning for Reinforcement Learning Based on Self-Pretraining for Strong Augmentation |
| title_sort | c2rl convolutional contrastive learning for reinforcement learning based on self pretraining for strong augmentation |
| topic | deep reinforcement learning self-supervised learning contrastive learning generalization data augmentation network randomization |
| url | https://www.mdpi.com/1424-8220/23/10/4946 |
| work_keys_str_mv | AT sanghoonpark c2rlconvolutionalcontrastivelearningforreinforcementlearningbasedonselfpretrainingforstrongaugmentation AT jihunkim c2rlconvolutionalcontrastivelearningforreinforcementlearningbasedonselfpretrainingforstrongaugmentation AT hanyoujeong c2rlconvolutionalcontrastivelearningforreinforcementlearningbasedonselfpretrainingforstrongaugmentation AT taekyoungkim c2rlconvolutionalcontrastivelearningforreinforcementlearningbasedonselfpretrainingforstrongaugmentation AT jinwooyoo c2rlconvolutionalcontrastivelearningforreinforcementlearningbasedonselfpretrainingforstrongaugmentation |