A Novel Deep Reinforcement Learning Based Framework for Gait Adjustment
Nowadays, millions of patients suffer from physical disabilities, including lower-limb disabilities. Researchers have adopted a variety of physical therapies based on the lower-limb exoskeleton, in which it is difficult to adjust equipment parameters in a timely fashion. Therefore, intelligent contr...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2022-12-01
|
| Series: | Mathematics |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2227-7390/11/1/178 |
| _version_ | 1797406758535168000 |
|---|---|
| author | Ang Li Jianping Chen Qiming Fu Hongjie Wu Yunzhe Wang You Lu |
| author_facet | Ang Li Jianping Chen Qiming Fu Hongjie Wu Yunzhe Wang You Lu |
| author_sort | Ang Li |
| collection | DOAJ |
| description | Nowadays, millions of patients suffer from physical disabilities, including lower-limb disabilities. Researchers have adopted a variety of physical therapies based on the lower-limb exoskeleton, in which it is difficult to adjust equipment parameters in a timely fashion. Therefore, intelligent control methods, for example, deep reinforcement learning (DRL), have been used to control the medical equipment used in human gait adjustment. In this study, based on the key-value attention mechanism, we reconstructed the agent’s observations by capturing the self-dependent feature information for decision-making in regard to each state sampled from the replay buffer. Moreover, based on Softmax Deep Double Deterministic policy gradients (SD3), a novel DRL-based framework, key-value attention-based SD3 (AT_SD3), has been proposed for gait adjustment. We demonstrated the effectiveness of our proposed framework in gait adjustment by comparing different gait trajectories, including the desired trajectory and the adjusted trajectory. The results showed that the simulated trajectories were closer to the desired trajectory, both in their shapes and values. Furthermore, by comparing the results of our experiments with those of other state-of-the-art methods, the results proved that our proposed framework exhibited better performance. |
| first_indexed | 2024-03-09T03:31:14Z |
| format | Article |
| id | doaj.art-29ef1c031b6b4b59837d97d4c64916d6 |
| institution | Directory Open Access Journal |
| issn | 2227-7390 |
| language | English |
| last_indexed | 2024-03-09T03:31:14Z |
| publishDate | 2022-12-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Mathematics |
| spelling | doaj.art-29ef1c031b6b4b59837d97d4c64916d62023-12-03T14:55:29ZengMDPI AGMathematics2227-73902022-12-0111117810.3390/math11010178A Novel Deep Reinforcement Learning Based Framework for Gait AdjustmentAng Li0Jianping Chen1Qiming Fu2Hongjie Wu3Yunzhe Wang4You Lu5School of Electronic and Information Engineering, Suzhou University of Science and Technology, Suzhou 215009, ChinaJiangsu Province Key Laboratory of Intelligent Building Energy Efficiency, Suzhou University of Science and Technology, Suzhou 215009, ChinaSchool of Electronic and Information Engineering, Suzhou University of Science and Technology, Suzhou 215009, ChinaSchool of Electronic and Information Engineering, Suzhou University of Science and Technology, Suzhou 215009, ChinaSchool of Electronic and Information Engineering, Suzhou University of Science and Technology, Suzhou 215009, ChinaSchool of Electronic and Information Engineering, Suzhou University of Science and Technology, Suzhou 215009, ChinaNowadays, millions of patients suffer from physical disabilities, including lower-limb disabilities. Researchers have adopted a variety of physical therapies based on the lower-limb exoskeleton, in which it is difficult to adjust equipment parameters in a timely fashion. Therefore, intelligent control methods, for example, deep reinforcement learning (DRL), have been used to control the medical equipment used in human gait adjustment. In this study, based on the key-value attention mechanism, we reconstructed the agent’s observations by capturing the self-dependent feature information for decision-making in regard to each state sampled from the replay buffer. Moreover, based on Softmax Deep Double Deterministic policy gradients (SD3), a novel DRL-based framework, key-value attention-based SD3 (AT_SD3), has been proposed for gait adjustment. We demonstrated the effectiveness of our proposed framework in gait adjustment by comparing different gait trajectories, including the desired trajectory and the adjusted trajectory. The results showed that the simulated trajectories were closer to the desired trajectory, both in their shapes and values. Furthermore, by comparing the results of our experiments with those of other state-of-the-art methods, the results proved that our proposed framework exhibited better performance.https://www.mdpi.com/2227-7390/11/1/178deep reinforcement learningattention mechanismstate reconstructiongait adjustment |
| spellingShingle | Ang Li Jianping Chen Qiming Fu Hongjie Wu Yunzhe Wang You Lu A Novel Deep Reinforcement Learning Based Framework for Gait Adjustment Mathematics deep reinforcement learning attention mechanism state reconstruction gait adjustment |
| title | A Novel Deep Reinforcement Learning Based Framework for Gait Adjustment |
| title_full | A Novel Deep Reinforcement Learning Based Framework for Gait Adjustment |
| title_fullStr | A Novel Deep Reinforcement Learning Based Framework for Gait Adjustment |
| title_full_unstemmed | A Novel Deep Reinforcement Learning Based Framework for Gait Adjustment |
| title_short | A Novel Deep Reinforcement Learning Based Framework for Gait Adjustment |
| title_sort | novel deep reinforcement learning based framework for gait adjustment |
| topic | deep reinforcement learning attention mechanism state reconstruction gait adjustment |
| url | https://www.mdpi.com/2227-7390/11/1/178 |
| work_keys_str_mv | AT angli anoveldeepreinforcementlearningbasedframeworkforgaitadjustment AT jianpingchen anoveldeepreinforcementlearningbasedframeworkforgaitadjustment AT qimingfu anoveldeepreinforcementlearningbasedframeworkforgaitadjustment AT hongjiewu anoveldeepreinforcementlearningbasedframeworkforgaitadjustment AT yunzhewang anoveldeepreinforcementlearningbasedframeworkforgaitadjustment AT youlu anoveldeepreinforcementlearningbasedframeworkforgaitadjustment AT angli noveldeepreinforcementlearningbasedframeworkforgaitadjustment AT jianpingchen noveldeepreinforcementlearningbasedframeworkforgaitadjustment AT qimingfu noveldeepreinforcementlearningbasedframeworkforgaitadjustment AT hongjiewu noveldeepreinforcementlearningbasedframeworkforgaitadjustment AT yunzhewang noveldeepreinforcementlearningbasedframeworkforgaitadjustment AT youlu noveldeepreinforcementlearningbasedframeworkforgaitadjustment |