A Hybrid End-to-End Control Strategy Combining Dueling Deep Q-network and PID for Transient Boost Control of a Diesel Engine with Variable Geometry Turbocharger and Cooled EGR
Deep reinforcement learning (DRL), which excels at solving a wide variety of Atari and board games, is an area of machine learning that combines the deep learning approach and reinforcement learning (RL). However, to the authors’ best knowledge, there seem to be few studies that apply the...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2019-09-01
|
| Series: | Energies |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1996-1073/12/19/3739 |
| _version_ | 1811187350432120832 |
|---|---|
| author | Bo Hu Jiaxi Li Shuang Li Jie Yang |
| author_facet | Bo Hu Jiaxi Li Shuang Li Jie Yang |
| author_sort | Bo Hu |
| collection | DOAJ |
| description | Deep reinforcement learning (DRL), which excels at solving a wide variety of Atari and board games, is an area of machine learning that combines the deep learning approach and reinforcement learning (RL). However, to the authors’ best knowledge, there seem to be few studies that apply the latest DRL algorithms on real-world powertrain control problems. If there are any, the requirement of classical model-free DRL algorithms typically for a large number of random exploration in order to realize good control performance makes it almost impossible to implement directly on a real plant. Unlike most of the other DRL studies, whose control strategies can only be trained in a simulation environment—especially when a control strategy has to be learned from scratch—in this study, a hybrid end-to-end control strategy combining one of the latest DRL approaches—i.e., a dueling deep Q-network and traditional Proportion Integration Differentiation (PID) controller—is built, assuming no fidelity simulation model exists. Taking the boost control of a diesel engine with a variable geometry turbocharger (VGT) and cooled (exhaust gas recirculation) EGR as an example, under the common driving cycle, the integral absolute error (IAE) values with the proposed algorithm are improved by 20.66% and 9.7% respectively for the control performance and generality index, compared with a fine-tuned PID benchmark. In addition, the proposed method can also improve system adaptiveness by adding another redundant control module. This makes it attractive to real plant control problems whose simulation models do not exist, and whose environment may change over time. |
| first_indexed | 2024-04-11T14:00:45Z |
| format | Article |
| id | doaj.art-aff430bb97dd486a9bcd72a68ebb2161 |
| institution | Directory Open Access Journal |
| issn | 1996-1073 |
| language | English |
| last_indexed | 2024-04-11T14:00:45Z |
| publishDate | 2019-09-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Energies |
| spelling | doaj.art-aff430bb97dd486a9bcd72a68ebb21612022-12-22T04:20:08ZengMDPI AGEnergies1996-10732019-09-011219373910.3390/en12193739en12193739A Hybrid End-to-End Control Strategy Combining Dueling Deep Q-network and PID for Transient Boost Control of a Diesel Engine with Variable Geometry Turbocharger and Cooled EGRBo Hu0Jiaxi Li1Shuang Li2Jie Yang3Key Laboratory of Advanced Manufacturing Technology for Automobile Parts, Ministry of Education, Chongqing University of Technology, Chongqing 400054, ChinaKey Laboratory of Advanced Manufacturing Technology for Automobile Parts, Ministry of Education, Chongqing University of Technology, Chongqing 400054, ChinaKey Laboratory of Advanced Manufacturing Technology for Automobile Parts, Ministry of Education, Chongqing University of Technology, Chongqing 400054, ChinaKey Laboratory of Advanced Manufacturing Technology for Automobile Parts, Ministry of Education, Chongqing University of Technology, Chongqing 400054, ChinaDeep reinforcement learning (DRL), which excels at solving a wide variety of Atari and board games, is an area of machine learning that combines the deep learning approach and reinforcement learning (RL). However, to the authors’ best knowledge, there seem to be few studies that apply the latest DRL algorithms on real-world powertrain control problems. If there are any, the requirement of classical model-free DRL algorithms typically for a large number of random exploration in order to realize good control performance makes it almost impossible to implement directly on a real plant. Unlike most of the other DRL studies, whose control strategies can only be trained in a simulation environment—especially when a control strategy has to be learned from scratch—in this study, a hybrid end-to-end control strategy combining one of the latest DRL approaches—i.e., a dueling deep Q-network and traditional Proportion Integration Differentiation (PID) controller—is built, assuming no fidelity simulation model exists. Taking the boost control of a diesel engine with a variable geometry turbocharger (VGT) and cooled (exhaust gas recirculation) EGR as an example, under the common driving cycle, the integral absolute error (IAE) values with the proposed algorithm are improved by 20.66% and 9.7% respectively for the control performance and generality index, compared with a fine-tuned PID benchmark. In addition, the proposed method can also improve system adaptiveness by adding another redundant control module. This makes it attractive to real plant control problems whose simulation models do not exist, and whose environment may change over time.https://www.mdpi.com/1996-1073/12/19/3739hybrid control strategydueling deep q-networkpidtransient boost controlvgt |
| spellingShingle | Bo Hu Jiaxi Li Shuang Li Jie Yang A Hybrid End-to-End Control Strategy Combining Dueling Deep Q-network and PID for Transient Boost Control of a Diesel Engine with Variable Geometry Turbocharger and Cooled EGR Energies hybrid control strategy dueling deep q-network pid transient boost control vgt |
| title | A Hybrid End-to-End Control Strategy Combining Dueling Deep Q-network and PID for Transient Boost Control of a Diesel Engine with Variable Geometry Turbocharger and Cooled EGR |
| title_full | A Hybrid End-to-End Control Strategy Combining Dueling Deep Q-network and PID for Transient Boost Control of a Diesel Engine with Variable Geometry Turbocharger and Cooled EGR |
| title_fullStr | A Hybrid End-to-End Control Strategy Combining Dueling Deep Q-network and PID for Transient Boost Control of a Diesel Engine with Variable Geometry Turbocharger and Cooled EGR |
| title_full_unstemmed | A Hybrid End-to-End Control Strategy Combining Dueling Deep Q-network and PID for Transient Boost Control of a Diesel Engine with Variable Geometry Turbocharger and Cooled EGR |
| title_short | A Hybrid End-to-End Control Strategy Combining Dueling Deep Q-network and PID for Transient Boost Control of a Diesel Engine with Variable Geometry Turbocharger and Cooled EGR |
| title_sort | hybrid end to end control strategy combining dueling deep q network and pid for transient boost control of a diesel engine with variable geometry turbocharger and cooled egr |
| topic | hybrid control strategy dueling deep q-network pid transient boost control vgt |
| url | https://www.mdpi.com/1996-1073/12/19/3739 |
| work_keys_str_mv | AT bohu ahybridendtoendcontrolstrategycombiningduelingdeepqnetworkandpidfortransientboostcontrolofadieselenginewithvariablegeometryturbochargerandcooledegr AT jiaxili ahybridendtoendcontrolstrategycombiningduelingdeepqnetworkandpidfortransientboostcontrolofadieselenginewithvariablegeometryturbochargerandcooledegr AT shuangli ahybridendtoendcontrolstrategycombiningduelingdeepqnetworkandpidfortransientboostcontrolofadieselenginewithvariablegeometryturbochargerandcooledegr AT jieyang ahybridendtoendcontrolstrategycombiningduelingdeepqnetworkandpidfortransientboostcontrolofadieselenginewithvariablegeometryturbochargerandcooledegr AT bohu hybridendtoendcontrolstrategycombiningduelingdeepqnetworkandpidfortransientboostcontrolofadieselenginewithvariablegeometryturbochargerandcooledegr AT jiaxili hybridendtoendcontrolstrategycombiningduelingdeepqnetworkandpidfortransientboostcontrolofadieselenginewithvariablegeometryturbochargerandcooledegr AT shuangli hybridendtoendcontrolstrategycombiningduelingdeepqnetworkandpidfortransientboostcontrolofadieselenginewithvariablegeometryturbochargerandcooledegr AT jieyang hybridendtoendcontrolstrategycombiningduelingdeepqnetworkandpidfortransientboostcontrolofadieselenginewithvariablegeometryturbochargerandcooledegr |