A Formal Verification Model for Performance Analysis of Reinforcement Learning Algorithms Applied t o Dynamic Networks
Routing data packets in a dynamic network is a difficult and important problem in computer networks. As the network is dynamic, it is subject to frequent topology changes and is subject to variable link costs due to congestion and bandwidth. Existing shortest path algorithms fail to converge to bett...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Stefan cel Mare University of Suceava
2017-04-01
|
| Series: | Journal of Applied Computer Science & Mathematics |
| Subjects: | |
| Online Access: | http://jacsm.ro/view/?pid=23_2 |
| _version_ | 1819127702138912768 |
|---|---|
| author | Shrirang Ambaji KULKARNI Raghavendra G . RAO |
| author_facet | Shrirang Ambaji KULKARNI Raghavendra G . RAO |
| author_sort | Shrirang Ambaji KULKARNI |
| collection | DOAJ |
| description | Routing data packets in a dynamic network is a difficult and important problem in computer networks. As the network is dynamic, it is subject to frequent topology changes and is subject to variable link costs due to congestion and bandwidth. Existing shortest path algorithms fail to converge to better solutions under dynamic network conditions. Reinforcement learning algorithms posses better adaptation techniques in dynamic environments. In this paper we apply model based Q-Routing technique for routing in dynamic network. To analyze the correctness of Q-Routing algorithms mathematically, we provide a proof and also implement a SPIN based verification model. We also perform simulation based analysis of Q-Routing for given metrics. |
| first_indexed | 2024-12-22T08:16:07Z |
| format | Article |
| id | doaj.art-3172dc72cd4b4a139f5886d6e31ec999 |
| institution | Directory Open Access Journal |
| issn | 2066-4273 2066-3129 |
| language | English |
| last_indexed | 2024-12-22T08:16:07Z |
| publishDate | 2017-04-01 |
| publisher | Stefan cel Mare University of Suceava |
| record_format | Article |
| series | Journal of Applied Computer Science & Mathematics |
| spelling | doaj.art-3172dc72cd4b4a139f5886d6e31ec9992022-12-21T18:32:53ZengStefan cel Mare University of SuceavaJournal of Applied Computer Science & Mathematics2066-42732066-31292017-04-01111131610.4316/JACSM.201701002A Formal Verification Model for Performance Analysis of Reinforcement Learning Algorithms Applied t o Dynamic NetworksShrirang Ambaji KULKARNI0Raghavendra G . RAO1GIT, Belgaum - 590008, IndiaNIE Institute of Technology, Mysore - 57008, IndiaRouting data packets in a dynamic network is a difficult and important problem in computer networks. As the network is dynamic, it is subject to frequent topology changes and is subject to variable link costs due to congestion and bandwidth. Existing shortest path algorithms fail to converge to better solutions under dynamic network conditions. Reinforcement learning algorithms posses better adaptation techniques in dynamic environments. In this paper we apply model based Q-Routing technique for routing in dynamic network. To analyze the correctness of Q-Routing algorithms mathematically, we provide a proof and also implement a SPIN based verification model. We also perform simulation based analysis of Q-Routing for given metrics.http://jacsm.ro/view/?pid=23_2Reinforcement LearningQ - Routing. SPIN model checkingDynamic NetworksQ - Routing. SPIN model checking |
| spellingShingle | Shrirang Ambaji KULKARNI Raghavendra G . RAO A Formal Verification Model for Performance Analysis of Reinforcement Learning Algorithms Applied t o Dynamic Networks Journal of Applied Computer Science & Mathematics Reinforcement Learning Q - Routing. SPIN model checking Dynamic Networks Q - Routing. SPIN model checking |
| title | A Formal Verification Model for Performance Analysis of Reinforcement Learning Algorithms Applied t o Dynamic Networks |
| title_full | A Formal Verification Model for Performance Analysis of Reinforcement Learning Algorithms Applied t o Dynamic Networks |
| title_fullStr | A Formal Verification Model for Performance Analysis of Reinforcement Learning Algorithms Applied t o Dynamic Networks |
| title_full_unstemmed | A Formal Verification Model for Performance Analysis of Reinforcement Learning Algorithms Applied t o Dynamic Networks |
| title_short | A Formal Verification Model for Performance Analysis of Reinforcement Learning Algorithms Applied t o Dynamic Networks |
| title_sort | formal verification model for performance analysis of reinforcement learning algorithms applied t o dynamic networks |
| topic | Reinforcement Learning Q - Routing. SPIN model checking Dynamic Networks Q - Routing. SPIN model checking |
| url | http://jacsm.ro/view/?pid=23_2 |
| work_keys_str_mv | AT shrirangambajikulkarni aformalverificationmodelforperformanceanalysisofreinforcementlearningalgorithmsappliedtodynamicnetworks AT raghavendragrao aformalverificationmodelforperformanceanalysisofreinforcementlearningalgorithmsappliedtodynamicnetworks AT shrirangambajikulkarni formalverificationmodelforperformanceanalysisofreinforcementlearningalgorithmsappliedtodynamicnetworks AT raghavendragrao formalverificationmodelforperformanceanalysisofreinforcementlearningalgorithmsappliedtodynamicnetworks |