Energy management of hybrid electric vehicles based on inverse reinforcement learning
Many scholars have conducted research on reinforcement learning in energy management, and verified that reinforcement learning methods have certain advantages. The guiding direction of the interaction between the agent and environment is determined by the reward function. However, the current design...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Elsevier
2022-11-01
|
| Series: | Energy Reports |
| Subjects: | |
| Online Access: | http://www.sciencedirect.com/science/article/pii/S2352484722007314 |
| _version_ | 1797902047347998720 |
|---|---|
| author | Hengxu Lv Chunyang Qi Chuanxue Song Shixin Song Ruiqiang Zhang Feng Xiao |
| author_facet | Hengxu Lv Chunyang Qi Chuanxue Song Shixin Song Ruiqiang Zhang Feng Xiao |
| author_sort | Hengxu Lv |
| collection | DOAJ |
| description | Many scholars have conducted research on reinforcement learning in energy management, and verified that reinforcement learning methods have certain advantages. The guiding direction of the interaction between the agent and environment is determined by the reward function. However, the current design of the reward function still has drawbacks. In most reinforcement learning energy management strategies, the design of the reward function is subjective and empirical, and describing the intention of the expert objectively is difficult. However, this condition cannot guarantee that the agent learns the optimal driving strategy under a given reward function. To address these problems, an energy management strategy based on reverse reinforcement learning is proposed to obtain the reward function weight under the expert trajectory, and then it is used to guide the behavior of the engine agent and the battery agent. Finally, the revised weights are re-entered into the forward reinforcement learning training. From the engine operating point, fuel consumption value, SOC change curve, and reward training process, indicate that the algorithm has certain advantages. In summary, the fuel-saving effect of the proposed algorithm is improved between 5%–10%. |
| first_indexed | 2024-04-10T09:11:37Z |
| format | Article |
| id | doaj.art-f5ae2d791f8e42c08cdab11bee6a0ac4 |
| institution | Directory Open Access Journal |
| issn | 2352-4847 |
| language | English |
| last_indexed | 2024-04-10T09:11:37Z |
| publishDate | 2022-11-01 |
| publisher | Elsevier |
| record_format | Article |
| series | Energy Reports |
| spelling | doaj.art-f5ae2d791f8e42c08cdab11bee6a0ac42023-02-21T05:11:04ZengElsevierEnergy Reports2352-48472022-11-01852155224Energy management of hybrid electric vehicles based on inverse reinforcement learningHengxu Lv0Chunyang Qi1Chuanxue Song2Shixin Song3Ruiqiang Zhang4Feng Xiao5State Key Laboratory of Automotive Simulation and Control, Jilin University, Changchun 130022, ChinaState Key Laboratory of Automotive Simulation and Control, Jilin University, Changchun 130022, ChinaCollege of Automotive Engineering, Jilin University, Changchun, 130022, ChinaSchool of Mechanical and Aerospace Engineering, Jilin University, Changchun 130022, ChinaState Key Laboratory of Automotive Simulation and Control, Jilin University, Changchun 130022, ChinaState Key Laboratory of Automotive Simulation and Control, Jilin University, Changchun 130022, China; Corresponding author.Many scholars have conducted research on reinforcement learning in energy management, and verified that reinforcement learning methods have certain advantages. The guiding direction of the interaction between the agent and environment is determined by the reward function. However, the current design of the reward function still has drawbacks. In most reinforcement learning energy management strategies, the design of the reward function is subjective and empirical, and describing the intention of the expert objectively is difficult. However, this condition cannot guarantee that the agent learns the optimal driving strategy under a given reward function. To address these problems, an energy management strategy based on reverse reinforcement learning is proposed to obtain the reward function weight under the expert trajectory, and then it is used to guide the behavior of the engine agent and the battery agent. Finally, the revised weights are re-entered into the forward reinforcement learning training. From the engine operating point, fuel consumption value, SOC change curve, and reward training process, indicate that the algorithm has certain advantages. In summary, the fuel-saving effect of the proposed algorithm is improved between 5%–10%.http://www.sciencedirect.com/science/article/pii/S2352484722007314Inverse reinforcement learningEnergy managementReward weight coefficientHybrid electric vehicle |
| spellingShingle | Hengxu Lv Chunyang Qi Chuanxue Song Shixin Song Ruiqiang Zhang Feng Xiao Energy management of hybrid electric vehicles based on inverse reinforcement learning Energy Reports Inverse reinforcement learning Energy management Reward weight coefficient Hybrid electric vehicle |
| title | Energy management of hybrid electric vehicles based on inverse reinforcement learning |
| title_full | Energy management of hybrid electric vehicles based on inverse reinforcement learning |
| title_fullStr | Energy management of hybrid electric vehicles based on inverse reinforcement learning |
| title_full_unstemmed | Energy management of hybrid electric vehicles based on inverse reinforcement learning |
| title_short | Energy management of hybrid electric vehicles based on inverse reinforcement learning |
| title_sort | energy management of hybrid electric vehicles based on inverse reinforcement learning |
| topic | Inverse reinforcement learning Energy management Reward weight coefficient Hybrid electric vehicle |
| url | http://www.sciencedirect.com/science/article/pii/S2352484722007314 |
| work_keys_str_mv | AT hengxulv energymanagementofhybridelectricvehiclesbasedoninversereinforcementlearning AT chunyangqi energymanagementofhybridelectricvehiclesbasedoninversereinforcementlearning AT chuanxuesong energymanagementofhybridelectricvehiclesbasedoninversereinforcementlearning AT shixinsong energymanagementofhybridelectricvehiclesbasedoninversereinforcementlearning AT ruiqiangzhang energymanagementofhybridelectricvehiclesbasedoninversereinforcementlearning AT fengxiao energymanagementofhybridelectricvehiclesbasedoninversereinforcementlearning |