Intelligent decision-making method for coal caving based on fuzzy deep Q-network
During the coal caving process in the fully mechanized caving face, due to the impact of coal dust and dust water mist on the workers' line of sight, there are problems of over-caving and under-caving in manually controlled coal caving. In order to solve this problem, the tail beam of the hydra...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | zho |
| Published: |
Editorial Department of Industry and Mine Automation
2023-04-01
|
| Series: | Gong-kuang zidonghua |
| Subjects: | |
| Online Access: | http://www.gkzdh.cn/article/doi/10.13272/j.issn.1671-251x.2022090068 |
| _version_ | 1797821431845748736 |
|---|---|
| author | YANG Yi WANG Shengwen CUI Kefei FEI Shumin |
| author_facet | YANG Yi WANG Shengwen CUI Kefei FEI Shumin |
| author_sort | YANG Yi |
| collection | DOAJ |
| description | During the coal caving process in the fully mechanized caving face, due to the impact of coal dust and dust water mist on the workers' line of sight, there are problems of over-caving and under-caving in manually controlled coal caving. In order to solve this problem, the tail beam of the hydraulic support is regarded as an intelligent agent, and the coal caving process is abstracted as a Markov optimal decision. A deep Q-network (DQN) is used to make decisions on the action of the coal drawing port. However, there is an overestimation problem in the DQN algorithm. A fuzzy deep Q-network (FDQN) algorithm is proposed and applied to intelligent decision-making of coal caving. The fuzzy control system is constructed by using the fuzzy features of the coal seam status in the coal caving process. The coal quantity and the coal gangue ratio in the coal seam state are taken as the inputs of the fuzzy control system. The output action of the fuzzy control system is replaced with the action of the DQN algorithm using the max operation to select the output Q value of the target network. It improves the online learning rate of the agent and increases the reward value of coal caving action. The coal caving model for the fully mechanized caving face is constructed. The three-dimensional numerical simulation of the coal caving process based on DQN, double depth Q-network (DDQN), and FDQN algorithms is conducted respectively. The results show that the FDQN algorithm has the fastest convergence speed, which is 31.6% faster than the DQN algorithm. It increases the online learning rate of the intelligent agent. The coal caving effect based on the FDQN algorithm is the best from three aspects: the straightness of the coal gangue boundary, the remaining coal above the tail beam, and the amount of gangue in the released body. The extraction rate based on the FDQN algorithm is the highest and the gangue content is the lowest. Compared with the DQN algorithm and DDQN algorithm, the extraction rate of the FDQN algorithm has increased by 2.8% and 0.7% respectively, and the gangue content has decreased by 2.1% and 13.2% respectively. The FDQN-based intelligent decision-making method for coal caving can adjust the action of the hydraulic support tail beam based on the coal seam occurrence status. It effectively solves the problems of over-caving and under-caving during the coal caving process. |
| first_indexed | 2024-03-13T09:52:37Z |
| format | Article |
| id | doaj.art-8186d812ba1148e49ad83bc8f106c502 |
| institution | Directory Open Access Journal |
| issn | 1671-251X |
| language | zho |
| last_indexed | 2024-03-13T09:52:37Z |
| publishDate | 2023-04-01 |
| publisher | Editorial Department of Industry and Mine Automation |
| record_format | Article |
| series | Gong-kuang zidonghua |
| spelling | doaj.art-8186d812ba1148e49ad83bc8f106c5022023-05-24T06:23:30ZzhoEditorial Department of Industry and Mine AutomationGong-kuang zidonghua1671-251X2023-04-01494788510.13272/j.issn.1671-251x.2022090068Intelligent decision-making method for coal caving based on fuzzy deep Q-networkYANG YiWANG ShengwenCUI KefeiFEI Shumin0School of Automation, Southeast University, Nanjing 210096, ChinaDuring the coal caving process in the fully mechanized caving face, due to the impact of coal dust and dust water mist on the workers' line of sight, there are problems of over-caving and under-caving in manually controlled coal caving. In order to solve this problem, the tail beam of the hydraulic support is regarded as an intelligent agent, and the coal caving process is abstracted as a Markov optimal decision. A deep Q-network (DQN) is used to make decisions on the action of the coal drawing port. However, there is an overestimation problem in the DQN algorithm. A fuzzy deep Q-network (FDQN) algorithm is proposed and applied to intelligent decision-making of coal caving. The fuzzy control system is constructed by using the fuzzy features of the coal seam status in the coal caving process. The coal quantity and the coal gangue ratio in the coal seam state are taken as the inputs of the fuzzy control system. The output action of the fuzzy control system is replaced with the action of the DQN algorithm using the max operation to select the output Q value of the target network. It improves the online learning rate of the agent and increases the reward value of coal caving action. The coal caving model for the fully mechanized caving face is constructed. The three-dimensional numerical simulation of the coal caving process based on DQN, double depth Q-network (DDQN), and FDQN algorithms is conducted respectively. The results show that the FDQN algorithm has the fastest convergence speed, which is 31.6% faster than the DQN algorithm. It increases the online learning rate of the intelligent agent. The coal caving effect based on the FDQN algorithm is the best from three aspects: the straightness of the coal gangue boundary, the remaining coal above the tail beam, and the amount of gangue in the released body. The extraction rate based on the FDQN algorithm is the highest and the gangue content is the lowest. Compared with the DQN algorithm and DDQN algorithm, the extraction rate of the FDQN algorithm has increased by 2.8% and 0.7% respectively, and the gangue content has decreased by 2.1% and 13.2% respectively. The FDQN-based intelligent decision-making method for coal caving can adjust the action of the hydraulic support tail beam based on the coal seam occurrence status. It effectively solves the problems of over-caving and under-caving during the coal caving process.http://www.gkzdh.cn/article/doi/10.13272/j.issn.1671-251x.2022090068fully mechanized caving faceintelligent coal cavingdeep reinforcement learningfuzzy deep q-networkfuzzy controlmarkov |
| spellingShingle | YANG Yi WANG Shengwen CUI Kefei FEI Shumin Intelligent decision-making method for coal caving based on fuzzy deep Q-network Gong-kuang zidonghua fully mechanized caving face intelligent coal caving deep reinforcement learning fuzzy deep q-network fuzzy control markov |
| title | Intelligent decision-making method for coal caving based on fuzzy deep Q-network |
| title_full | Intelligent decision-making method for coal caving based on fuzzy deep Q-network |
| title_fullStr | Intelligent decision-making method for coal caving based on fuzzy deep Q-network |
| title_full_unstemmed | Intelligent decision-making method for coal caving based on fuzzy deep Q-network |
| title_short | Intelligent decision-making method for coal caving based on fuzzy deep Q-network |
| title_sort | intelligent decision making method for coal caving based on fuzzy deep q network |
| topic | fully mechanized caving face intelligent coal caving deep reinforcement learning fuzzy deep q-network fuzzy control markov |
| url | http://www.gkzdh.cn/article/doi/10.13272/j.issn.1671-251x.2022090068 |
| work_keys_str_mv | AT yangyi intelligentdecisionmakingmethodforcoalcavingbasedonfuzzydeepqnetwork AT wangshengwen intelligentdecisionmakingmethodforcoalcavingbasedonfuzzydeepqnetwork AT cuikefei intelligentdecisionmakingmethodforcoalcavingbasedonfuzzydeepqnetwork AT feishumin intelligentdecisionmakingmethodforcoalcavingbasedonfuzzydeepqnetwork |