Model-Based Reinforcement Learning with Automated Planning for Network Management
Reinforcement Learning (RL) comes with the promise of automating network management. However, due to its trial-and-error learning approach, model-based RL (MBRL) is not applicable in some network management scenarios. This paper explores the potential of using Automated Planning (AP) to achieve this...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2022-08-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/22/16/6301 |
| _version_ | 1797442003913408512 |
|---|---|
| author | Armando Ordonez Oscar Mauricio Caicedo William Villota Angela Rodriguez-Vivas Nelson L. S. da Fonseca |
| author_facet | Armando Ordonez Oscar Mauricio Caicedo William Villota Angela Rodriguez-Vivas Nelson L. S. da Fonseca |
| author_sort | Armando Ordonez |
| collection | DOAJ |
| description | Reinforcement Learning (RL) comes with the promise of automating network management. However, due to its trial-and-error learning approach, model-based RL (MBRL) is not applicable in some network management scenarios. This paper explores the potential of using Automated Planning (AP) to achieve this MBRL in the functional areas of network management. In addition, a comparison of several integration strategies of AP and RL is depicted. We also describe an architecture that realizes a cognitive management control loop by combining AP and RL. Our experiments evaluate on a simulated environment evidence that the combination proposed improves model-free RL but demonstrates lower performance than Deep RL regarding the reward and convergence time metrics. Nonetheless, AP-based MBRL is useful when the prediction model needs to be understood and when the high computational complexity of Deep RL can not be used. |
| first_indexed | 2024-03-09T12:35:31Z |
| format | Article |
| id | doaj.art-02dc1b518ab644b290bebd14b6b24c0e |
| institution | Directory Open Access Journal |
| issn | 1424-8220 |
| language | English |
| last_indexed | 2024-03-09T12:35:31Z |
| publishDate | 2022-08-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj.art-02dc1b518ab644b290bebd14b6b24c0e2023-11-30T22:24:34ZengMDPI AGSensors1424-82202022-08-012216630110.3390/s22166301Model-Based Reinforcement Learning with Automated Planning for Network ManagementArmando Ordonez0Oscar Mauricio Caicedo1William Villota2Angela Rodriguez-Vivas3Nelson L. S. da Fonseca4Universidad ICESI, Cali 760031, ColombiaDepartamento de Telematica, Universidad del Cauca, Popayan 190002, ColombiaInstitute of Computing, University of Campinas, Campinas 13083-852, BrazilDepartamento de Telematica, Universidad del Cauca, Popayan 190002, ColombiaInstitute of Computing, University of Campinas, Campinas 13083-852, BrazilReinforcement Learning (RL) comes with the promise of automating network management. However, due to its trial-and-error learning approach, model-based RL (MBRL) is not applicable in some network management scenarios. This paper explores the potential of using Automated Planning (AP) to achieve this MBRL in the functional areas of network management. In addition, a comparison of several integration strategies of AP and RL is depicted. We also describe an architecture that realizes a cognitive management control loop by combining AP and RL. Our experiments evaluate on a simulated environment evidence that the combination proposed improves model-free RL but demonstrates lower performance than Deep RL regarding the reward and convergence time metrics. Nonetheless, AP-based MBRL is useful when the prediction model needs to be understood and when the high computational complexity of Deep RL can not be used.https://www.mdpi.com/1424-8220/22/16/6301automated planningmodel basedreinforcement learningnetwork management |
| spellingShingle | Armando Ordonez Oscar Mauricio Caicedo William Villota Angela Rodriguez-Vivas Nelson L. S. da Fonseca Model-Based Reinforcement Learning with Automated Planning for Network Management Sensors automated planning model based reinforcement learning network management |
| title | Model-Based Reinforcement Learning with Automated Planning for Network Management |
| title_full | Model-Based Reinforcement Learning with Automated Planning for Network Management |
| title_fullStr | Model-Based Reinforcement Learning with Automated Planning for Network Management |
| title_full_unstemmed | Model-Based Reinforcement Learning with Automated Planning for Network Management |
| title_short | Model-Based Reinforcement Learning with Automated Planning for Network Management |
| title_sort | model based reinforcement learning with automated planning for network management |
| topic | automated planning model based reinforcement learning network management |
| url | https://www.mdpi.com/1424-8220/22/16/6301 |
| work_keys_str_mv | AT armandoordonez modelbasedreinforcementlearningwithautomatedplanningfornetworkmanagement AT oscarmauriciocaicedo modelbasedreinforcementlearningwithautomatedplanningfornetworkmanagement AT williamvillota modelbasedreinforcementlearningwithautomatedplanningfornetworkmanagement AT angelarodriguezvivas modelbasedreinforcementlearningwithautomatedplanningfornetworkmanagement AT nelsonlsdafonseca modelbasedreinforcementlearningwithautomatedplanningfornetworkmanagement |