Design and Comparison of Reinforcement-Learning-Based Time-Varying PID Controllers with Gain-Scheduled Actions
This paper presents innovative reinforcement learning methods for automatically tuning the parameters of a proportional integral derivative controller. Conventionally, the high dimension of the Q-table is a primary drawback when implementing a reinforcement learning algorithm. To overcome the obstac...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2021-11-01
|
| Series: | Machines |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2075-1702/9/12/319 |
| _version_ | 1797502845162881024 |
|---|---|
| author | Yi-Liang Yeh Po-Kai Yang |
| author_facet | Yi-Liang Yeh Po-Kai Yang |
| author_sort | Yi-Liang Yeh |
| collection | DOAJ |
| description | This paper presents innovative reinforcement learning methods for automatically tuning the parameters of a proportional integral derivative controller. Conventionally, the high dimension of the Q-table is a primary drawback when implementing a reinforcement learning algorithm. To overcome the obstacle, the idea underlying the <i>n</i>-armed bandit problem is used in this paper. Moreover, gain-scheduled actions are presented to tune the algorithms to improve the overall system behavior; therefore, the proposed controllers fulfill the multiple performance requirements. An experiment was conducted for the piezo-actuated stage to illustrate the effectiveness of the proposed control designs relative to competing algorithms. |
| first_indexed | 2024-03-10T03:41:57Z |
| format | Article |
| id | doaj.art-5d9cb1b57b5f4b2eb52fd89f2fff49b2 |
| institution | Directory Open Access Journal |
| issn | 2075-1702 |
| language | English |
| last_indexed | 2024-03-10T03:41:57Z |
| publishDate | 2021-11-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Machines |
| spelling | doaj.art-5d9cb1b57b5f4b2eb52fd89f2fff49b22023-11-23T09:16:29ZengMDPI AGMachines2075-17022021-11-0191231910.3390/machines9120319Design and Comparison of Reinforcement-Learning-Based Time-Varying PID Controllers with Gain-Scheduled ActionsYi-Liang Yeh0Po-Kai Yang1Department of Mechanical Engineering, National Taipei University of Technology, Taipei 106344, TaiwanDepartment of Mechanical Engineering, National Taipei University of Technology, Taipei 106344, TaiwanThis paper presents innovative reinforcement learning methods for automatically tuning the parameters of a proportional integral derivative controller. Conventionally, the high dimension of the Q-table is a primary drawback when implementing a reinforcement learning algorithm. To overcome the obstacle, the idea underlying the <i>n</i>-armed bandit problem is used in this paper. Moreover, gain-scheduled actions are presented to tune the algorithms to improve the overall system behavior; therefore, the proposed controllers fulfill the multiple performance requirements. An experiment was conducted for the piezo-actuated stage to illustrate the effectiveness of the proposed control designs relative to competing algorithms.https://www.mdpi.com/2075-1702/9/12/319reinforcement learningQ-learningSarsagain-scheduled actiontime-varying PID controllerPZT stage |
| spellingShingle | Yi-Liang Yeh Po-Kai Yang Design and Comparison of Reinforcement-Learning-Based Time-Varying PID Controllers with Gain-Scheduled Actions Machines reinforcement learning Q-learning Sarsa gain-scheduled action time-varying PID controller PZT stage |
| title | Design and Comparison of Reinforcement-Learning-Based Time-Varying PID Controllers with Gain-Scheduled Actions |
| title_full | Design and Comparison of Reinforcement-Learning-Based Time-Varying PID Controllers with Gain-Scheduled Actions |
| title_fullStr | Design and Comparison of Reinforcement-Learning-Based Time-Varying PID Controllers with Gain-Scheduled Actions |
| title_full_unstemmed | Design and Comparison of Reinforcement-Learning-Based Time-Varying PID Controllers with Gain-Scheduled Actions |
| title_short | Design and Comparison of Reinforcement-Learning-Based Time-Varying PID Controllers with Gain-Scheduled Actions |
| title_sort | design and comparison of reinforcement learning based time varying pid controllers with gain scheduled actions |
| topic | reinforcement learning Q-learning Sarsa gain-scheduled action time-varying PID controller PZT stage |
| url | https://www.mdpi.com/2075-1702/9/12/319 |
| work_keys_str_mv | AT yiliangyeh designandcomparisonofreinforcementlearningbasedtimevaryingpidcontrollerswithgainscheduledactions AT pokaiyang designandcomparisonofreinforcementlearningbasedtimevaryingpidcontrollerswithgainscheduledactions |