A Routing Optimization Method for Software-Defined Optical Transport Networks Based on Ensembles and Reinforcement Learning
Optical transport networks (OTNs) are widely used in backbone- and metro-area transmission networks to increase network transmission capacity. In the OTN, it is particularly crucial to rationally allocate routes and maximize network capacities. By employing deep reinforcement learning (DRL)- and sof...
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2022-10-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/22/21/8139 |
| _version_ | 1797466547933937664 |
|---|---|
| author | Junyan Chen Wei Xiao Xinmei Li Yang Zheng Xuefeng Huang Danli Huang Min Wang |
| author_facet | Junyan Chen Wei Xiao Xinmei Li Yang Zheng Xuefeng Huang Danli Huang Min Wang |
| author_sort | Junyan Chen |
| collection | DOAJ |
| description | Optical transport networks (OTNs) are widely used in backbone- and metro-area transmission networks to increase network transmission capacity. In the OTN, it is particularly crucial to rationally allocate routes and maximize network capacities. By employing deep reinforcement learning (DRL)- and software-defined networking (SDN)-based solutions, the capacity of optical networks can be effectively increased. However, because most DRL-based routing optimization methods have low sample usage and difficulty in coping with sudden network connectivity changes, converging in software-defined OTN scenarios is challenging. Additionally, the generalization ability of these methods is weak. This paper proposes an ensembles- and message-passing neural-network-based Deep Q-Network (EMDQN) method for optical network routing optimization to address this problem. To effectively explore the environment and improve agent performance, the multiple EMDQN agents select actions based on the highest upper-confidence bounds. Furthermore, the EMDQN agent captures the network’s spatial feature information using a message passing neural network (MPNN)-based DRL policy network, which enables the DRL agent to have generalization capability. The experimental results show that the EMDQN algorithm proposed in this paper performs better in terms of convergence. EMDQN effectively improves the throughput rate and link utilization of optical networks and has better generalization capabilities. |
| first_indexed | 2024-03-09T18:41:20Z |
| format | Article |
| id | doaj.art-b813c80bda9d461091c907b4a72a66fa |
| institution | Directory Open Access Journal |
| issn | 1424-8220 |
| language | English |
| last_indexed | 2024-03-09T18:41:20Z |
| publishDate | 2022-10-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj.art-b813c80bda9d461091c907b4a72a66fa2023-11-24T06:43:19ZengMDPI AGSensors1424-82202022-10-012221813910.3390/s22218139A Routing Optimization Method for Software-Defined Optical Transport Networks Based on Ensembles and Reinforcement LearningJunyan Chen0Wei Xiao1Xinmei Li2Yang Zheng3Xuefeng Huang4Danli Huang5Min Wang6School of Computer Science and Information Security, Guilin University of Electronic Technology, Guilin 541004, ChinaSchool of Computer Science and Information Security, Guilin University of Electronic Technology, Guilin 541004, ChinaSchool of Computer Science and Information Security, Guilin University of Electronic Technology, Guilin 541004, ChinaInstitute of Automation, Chinese Academy of Sciences, Beijing 100190, ChinaSchool of Computer Science and Information Security, Guilin University of Electronic Technology, Guilin 541004, ChinaSchool of Computer Science and Information Security, Guilin University of Electronic Technology, Guilin 541004, ChinaSchool of Computer Science and Information Security, Guilin University of Electronic Technology, Guilin 541004, ChinaOptical transport networks (OTNs) are widely used in backbone- and metro-area transmission networks to increase network transmission capacity. In the OTN, it is particularly crucial to rationally allocate routes and maximize network capacities. By employing deep reinforcement learning (DRL)- and software-defined networking (SDN)-based solutions, the capacity of optical networks can be effectively increased. However, because most DRL-based routing optimization methods have low sample usage and difficulty in coping with sudden network connectivity changes, converging in software-defined OTN scenarios is challenging. Additionally, the generalization ability of these methods is weak. This paper proposes an ensembles- and message-passing neural-network-based Deep Q-Network (EMDQN) method for optical network routing optimization to address this problem. To effectively explore the environment and improve agent performance, the multiple EMDQN agents select actions based on the highest upper-confidence bounds. Furthermore, the EMDQN agent captures the network’s spatial feature information using a message passing neural network (MPNN)-based DRL policy network, which enables the DRL agent to have generalization capability. The experimental results show that the EMDQN algorithm proposed in this paper performs better in terms of convergence. EMDQN effectively improves the throughput rate and link utilization of optical networks and has better generalization capabilities.https://www.mdpi.com/1424-8220/22/21/8139optical transport networksoftware-defined networkingdeep Q-networkmessage-passing neural networkensemble learning |
| spellingShingle | Junyan Chen Wei Xiao Xinmei Li Yang Zheng Xuefeng Huang Danli Huang Min Wang A Routing Optimization Method for Software-Defined Optical Transport Networks Based on Ensembles and Reinforcement Learning Sensors optical transport network software-defined networking deep Q-network message-passing neural network ensemble learning |
| title | A Routing Optimization Method for Software-Defined Optical Transport Networks Based on Ensembles and Reinforcement Learning |
| title_full | A Routing Optimization Method for Software-Defined Optical Transport Networks Based on Ensembles and Reinforcement Learning |
| title_fullStr | A Routing Optimization Method for Software-Defined Optical Transport Networks Based on Ensembles and Reinforcement Learning |
| title_full_unstemmed | A Routing Optimization Method for Software-Defined Optical Transport Networks Based on Ensembles and Reinforcement Learning |
| title_short | A Routing Optimization Method for Software-Defined Optical Transport Networks Based on Ensembles and Reinforcement Learning |
| title_sort | routing optimization method for software defined optical transport networks based on ensembles and reinforcement learning |
| topic | optical transport network software-defined networking deep Q-network message-passing neural network ensemble learning |
| url | https://www.mdpi.com/1424-8220/22/21/8139 |
| work_keys_str_mv | AT junyanchen aroutingoptimizationmethodforsoftwaredefinedopticaltransportnetworksbasedonensemblesandreinforcementlearning AT weixiao aroutingoptimizationmethodforsoftwaredefinedopticaltransportnetworksbasedonensemblesandreinforcementlearning AT xinmeili aroutingoptimizationmethodforsoftwaredefinedopticaltransportnetworksbasedonensemblesandreinforcementlearning AT yangzheng aroutingoptimizationmethodforsoftwaredefinedopticaltransportnetworksbasedonensemblesandreinforcementlearning AT xuefenghuang aroutingoptimizationmethodforsoftwaredefinedopticaltransportnetworksbasedonensemblesandreinforcementlearning AT danlihuang aroutingoptimizationmethodforsoftwaredefinedopticaltransportnetworksbasedonensemblesandreinforcementlearning AT minwang aroutingoptimizationmethodforsoftwaredefinedopticaltransportnetworksbasedonensemblesandreinforcementlearning AT junyanchen routingoptimizationmethodforsoftwaredefinedopticaltransportnetworksbasedonensemblesandreinforcementlearning AT weixiao routingoptimizationmethodforsoftwaredefinedopticaltransportnetworksbasedonensemblesandreinforcementlearning AT xinmeili routingoptimizationmethodforsoftwaredefinedopticaltransportnetworksbasedonensemblesandreinforcementlearning AT yangzheng routingoptimizationmethodforsoftwaredefinedopticaltransportnetworksbasedonensemblesandreinforcementlearning AT xuefenghuang routingoptimizationmethodforsoftwaredefinedopticaltransportnetworksbasedonensemblesandreinforcementlearning AT danlihuang routingoptimizationmethodforsoftwaredefinedopticaltransportnetworksbasedonensemblesandreinforcementlearning AT minwang routingoptimizationmethodforsoftwaredefinedopticaltransportnetworksbasedonensemblesandreinforcementlearning |