Transfer learning through policy abstraction using learning vector quantization
Reinforcement learning (RL) enables an agent to find a solution to a problem by interacting with the environment. However, the learning process always starts from scratch and possibly takes a long time. Here, knowledge transfer between tasks is considered. In this paper, we argue that an abstraction...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
UTeM
2018
|
| Subjects: | |
| Online Access: | http://umpir.ump.edu.my/id/eprint/21033/1/Transfer%20learning%20through%20policy%20abstraction.pdf |
| _version_ | 1796992692626915328 |
|---|---|
| author | Ahmad Afif, Mohd Faudzi Takano, Hirotaka Murata, Jun'ichi |
| author_facet | Ahmad Afif, Mohd Faudzi Takano, Hirotaka Murata, Jun'ichi |
| author_sort | Ahmad Afif, Mohd Faudzi |
| collection | UMP |
| description | Reinforcement learning (RL) enables an agent to find a solution to a problem by interacting with the environment. However, the learning process always starts from scratch and possibly takes a long time. Here, knowledge transfer between tasks is considered. In this paper, we argue that an abstraction can improve the transfer learning. Modified learning vector quantization (LVQ) that can manipulate its network weights is proposed to perform an abstraction, an adaptation and a precaution. At first, the abstraction is performed by extracting an abstract policy out of a learned policy which is acquired through conventional RL method, Q-learning. The abstract policy then is used in a new task as prior information. Here, the adaptation or policy learning as well as new task's abstract policy generating are performed using only a single operation. Simulation results show that the representation of acquired abstract policy is interpretable, that the modified LVQ successfully performs policy learning as well as generates abstract policy and that the application of generalized common abstract policy produces better results by more effectively guiding the agent when learning a new task. |
| first_indexed | 2024-03-06T12:23:32Z |
| format | Article |
| id | UMPir21033 |
| institution | Universiti Malaysia Pahang |
| language | English |
| last_indexed | 2024-03-06T12:23:32Z |
| publishDate | 2018 |
| publisher | UTeM |
| record_format | dspace |
| spelling | UMPir210332018-08-20T07:10:21Z http://umpir.ump.edu.my/id/eprint/21033/ Transfer learning through policy abstraction using learning vector quantization Ahmad Afif, Mohd Faudzi Takano, Hirotaka Murata, Jun'ichi TK Electrical engineering. Electronics Nuclear engineering Reinforcement learning (RL) enables an agent to find a solution to a problem by interacting with the environment. However, the learning process always starts from scratch and possibly takes a long time. Here, knowledge transfer between tasks is considered. In this paper, we argue that an abstraction can improve the transfer learning. Modified learning vector quantization (LVQ) that can manipulate its network weights is proposed to perform an abstraction, an adaptation and a precaution. At first, the abstraction is performed by extracting an abstract policy out of a learned policy which is acquired through conventional RL method, Q-learning. The abstract policy then is used in a new task as prior information. Here, the adaptation or policy learning as well as new task's abstract policy generating are performed using only a single operation. Simulation results show that the representation of acquired abstract policy is interpretable, that the modified LVQ successfully performs policy learning as well as generates abstract policy and that the application of generalized common abstract policy produces better results by more effectively guiding the agent when learning a new task. UTeM 2018 Article PeerReviewed pdf en cc_by http://umpir.ump.edu.my/id/eprint/21033/1/Transfer%20learning%20through%20policy%20abstraction.pdf Ahmad Afif, Mohd Faudzi and Takano, Hirotaka and Murata, Jun'ichi (2018) Transfer learning through policy abstraction using learning vector quantization. Journal of Telecommunication, Electronic and Computer Engineering, 10 (1-3). pp. 163-168. ISSN 2180-1843 (Print); 2289-8131 (Online). (Published) http://journal.utem.edu.my/index.php/jtec/article/view/3505/2453 |
| spellingShingle | TK Electrical engineering. Electronics Nuclear engineering Ahmad Afif, Mohd Faudzi Takano, Hirotaka Murata, Jun'ichi Transfer learning through policy abstraction using learning vector quantization |
| title | Transfer learning through policy abstraction using learning vector quantization |
| title_full | Transfer learning through policy abstraction using learning vector quantization |
| title_fullStr | Transfer learning through policy abstraction using learning vector quantization |
| title_full_unstemmed | Transfer learning through policy abstraction using learning vector quantization |
| title_short | Transfer learning through policy abstraction using learning vector quantization |
| title_sort | transfer learning through policy abstraction using learning vector quantization |
| topic | TK Electrical engineering. Electronics Nuclear engineering |
| url | http://umpir.ump.edu.my/id/eprint/21033/1/Transfer%20learning%20through%20policy%20abstraction.pdf |
| work_keys_str_mv | AT ahmadafifmohdfaudzi transferlearningthroughpolicyabstractionusinglearningvectorquantization AT takanohirotaka transferlearningthroughpolicyabstractionusinglearningvectorquantization AT muratajunichi transferlearningthroughpolicyabstractionusinglearningvectorquantization |