Research on Energy-Saving Routing Technology Based on Deep Reinforcement Learning
With the vigorous development of the Internet, the network traffic of data centers has exploded, and at the same time, the network energy consumption of data centers has also increased rapidly. Existing routing algorithms only realize routing optimization through Quality of Service (QoS) and Quality...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2022-06-01
|
| Series: | Electronics |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2079-9292/11/13/2035 |
| _version_ | 1797480420042866688 |
|---|---|
| author | Xiangyu Zheng Wanwei Huang Sunan Wang Jianwei Zhang Huanlong Zhang |
| author_facet | Xiangyu Zheng Wanwei Huang Sunan Wang Jianwei Zhang Huanlong Zhang |
| author_sort | Xiangyu Zheng |
| collection | DOAJ |
| description | With the vigorous development of the Internet, the network traffic of data centers has exploded, and at the same time, the network energy consumption of data centers has also increased rapidly. Existing routing algorithms only realize routing optimization through Quality of Service (QoS) and Quality of Experience (QoE), which ignores the energy consumption of data center networks. Aiming at this problem, this paper proposes an Ee-Routing algorithm, which is an energy-saving routing algorithm based on deep reinforcement learning. First, our method takes the energy consumption and network performance of the data plane in the software-defined network as the joint optimization goal and establishes an energy-efficient traffic scheduling scheme for the elephant flows and the mice flows. Then, we use Deep Deterministic Policy Gradient (DDPG), which is a deep learning framework, to achieve continuous and energy-efficient traffic scheduling for joint optimization goals. The training process of our method is based on a Convolutional Neural Network (CNN), which can effectively improve the convergence efficiency of the algorithm. After the algorithm training converges, the energy-efficient path weights of the elephant flows and the mice flows are output, and the balanced scheduling of routing energy-saving and network performance is completed. Finally, the results show that our algorithm has good convergence and stability. Compared with the DQN-EER routing algorithm, Ee-Routing improves the energy saving percentage by 13.93%, and compared with the EARS routing algorithm, Ee-Routing reduces the delay by 13.73%, increases the throughput by 10.91%, and reduces the packet loss rate by 13.51%. |
| first_indexed | 2024-03-09T21:59:45Z |
| format | Article |
| id | doaj.art-97b6c405befd4871b5ec57867dc5dc08 |
| institution | Directory Open Access Journal |
| issn | 2079-9292 |
| language | English |
| last_indexed | 2024-03-09T21:59:45Z |
| publishDate | 2022-06-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Electronics |
| spelling | doaj.art-97b6c405befd4871b5ec57867dc5dc082023-11-23T19:51:49ZengMDPI AGElectronics2079-92922022-06-011113203510.3390/electronics11132035Research on Energy-Saving Routing Technology Based on Deep Reinforcement LearningXiangyu Zheng0Wanwei Huang1Sunan Wang2Jianwei Zhang3Huanlong Zhang4College of Software Engineering, Zhengzhou University of Light Industry, Zhengzhou 450001, ChinaCollege of Software Engineering, Zhengzhou University of Light Industry, Zhengzhou 450001, ChinaSchool of Electronic and Communication Engineering, Shenzhen Polytechnic, Shenzhen 518055, ChinaCollege of Software Engineering, Zhengzhou University of Light Industry, Zhengzhou 450001, ChinaCollege of Software Engineering, Zhengzhou University of Light Industry, Zhengzhou 450001, ChinaWith the vigorous development of the Internet, the network traffic of data centers has exploded, and at the same time, the network energy consumption of data centers has also increased rapidly. Existing routing algorithms only realize routing optimization through Quality of Service (QoS) and Quality of Experience (QoE), which ignores the energy consumption of data center networks. Aiming at this problem, this paper proposes an Ee-Routing algorithm, which is an energy-saving routing algorithm based on deep reinforcement learning. First, our method takes the energy consumption and network performance of the data plane in the software-defined network as the joint optimization goal and establishes an energy-efficient traffic scheduling scheme for the elephant flows and the mice flows. Then, we use Deep Deterministic Policy Gradient (DDPG), which is a deep learning framework, to achieve continuous and energy-efficient traffic scheduling for joint optimization goals. The training process of our method is based on a Convolutional Neural Network (CNN), which can effectively improve the convergence efficiency of the algorithm. After the algorithm training converges, the energy-efficient path weights of the elephant flows and the mice flows are output, and the balanced scheduling of routing energy-saving and network performance is completed. Finally, the results show that our algorithm has good convergence and stability. Compared with the DQN-EER routing algorithm, Ee-Routing improves the energy saving percentage by 13.93%, and compared with the EARS routing algorithm, Ee-Routing reduces the delay by 13.73%, increases the throughput by 10.91%, and reduces the packet loss rate by 13.51%.https://www.mdpi.com/2079-9292/11/13/2035software-defined network (SDN)energy-efficient routingdeep deterministic policy gradient (DDPG)convolutional neural network (CNN) |
| spellingShingle | Xiangyu Zheng Wanwei Huang Sunan Wang Jianwei Zhang Huanlong Zhang Research on Energy-Saving Routing Technology Based on Deep Reinforcement Learning Electronics software-defined network (SDN) energy-efficient routing deep deterministic policy gradient (DDPG) convolutional neural network (CNN) |
| title | Research on Energy-Saving Routing Technology Based on Deep Reinforcement Learning |
| title_full | Research on Energy-Saving Routing Technology Based on Deep Reinforcement Learning |
| title_fullStr | Research on Energy-Saving Routing Technology Based on Deep Reinforcement Learning |
| title_full_unstemmed | Research on Energy-Saving Routing Technology Based on Deep Reinforcement Learning |
| title_short | Research on Energy-Saving Routing Technology Based on Deep Reinforcement Learning |
| title_sort | research on energy saving routing technology based on deep reinforcement learning |
| topic | software-defined network (SDN) energy-efficient routing deep deterministic policy gradient (DDPG) convolutional neural network (CNN) |
| url | https://www.mdpi.com/2079-9292/11/13/2035 |
| work_keys_str_mv | AT xiangyuzheng researchonenergysavingroutingtechnologybasedondeepreinforcementlearning AT wanweihuang researchonenergysavingroutingtechnologybasedondeepreinforcementlearning AT sunanwang researchonenergysavingroutingtechnologybasedondeepreinforcementlearning AT jianweizhang researchonenergysavingroutingtechnologybasedondeepreinforcementlearning AT huanlongzhang researchonenergysavingroutingtechnologybasedondeepreinforcementlearning |