Offloading Mechanisms Based on Reinforcement Learning and Deep Learning Algorithms in the Fog Computing Environment
Fog computing has emerged as a computing paradigm for resource-restricted Internet of things (IoT) devices to support time-sensitive and computationally intensive applications. Offloading can be utilized to transfer resource-intensive tasks from resource-limited end devices to a resource-rich fog or...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2023-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10035389/ |
| _version_ | 1811163489863991296 |
|---|---|
| author | Dezheen H. Abdulazeez Shavan K. Askar |
| author_facet | Dezheen H. Abdulazeez Shavan K. Askar |
| author_sort | Dezheen H. Abdulazeez |
| collection | DOAJ |
| description | Fog computing has emerged as a computing paradigm for resource-restricted Internet of things (IoT) devices to support time-sensitive and computationally intensive applications. Offloading can be utilized to transfer resource-intensive tasks from resource-limited end devices to a resource-rich fog or cloud layer to reduce end-to-end latency and enhance the performance of the system. However, this advantage is still challenging to achieve in systems with a high request rate because it leads to long queues of tasks in fog nodes and reveals inefficiencies in terms of delays. In this regard, reinforcement learning (RL) is a well-known method for addressing such decision-making issues. However, in large-scale wireless networks, both action and state spaces are complex and extremely extensive. Consequently, reinforcement learning techniques may not be able to identify an efficient strategy within an acceptable time frame. Hence, deep reinforcement learning (DRL) was developed to integrate RL and deep learning (DL) to address this problem. This paper presents a systematic analysis of using RL or DRL algorithms to address offloading-related issues in fog computing. First, the taxonomy of fog computing offloading mechanisms based on RL and DRL algorithms was divided into three major categories: value-based, policy-based, and hybrid-based algorithms. These categories were then compared based on important features, including offloading problem formulation, utilized techniques, performance metrics, evaluation tools, case studies, their strengths and drawbacks, offloading directions, offloading mode, SDN-based architecture, and offloading decisions. Finally, the future research directions and open issues are discussed thoroughly. |
| first_indexed | 2024-04-10T15:07:04Z |
| format | Article |
| id | doaj.art-027d48dc39144f02a45939c814f74eb8 |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-04-10T15:07:04Z |
| publishDate | 2023-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-027d48dc39144f02a45939c814f74eb82023-02-15T00:00:29ZengIEEEIEEE Access2169-35362023-01-0111125551258610.1109/ACCESS.2023.324188110035389Offloading Mechanisms Based on Reinforcement Learning and Deep Learning Algorithms in the Fog Computing EnvironmentDezheen H. Abdulazeez0https://orcid.org/0000-0002-6918-5600Shavan K. Askar1https://orcid.org/0000-0002-9279-8181Department of Computer Science, University of Duhok, Duhok, IraqErbil Technical Engineering College, Erbil Polytechnic University, Erbil, IraqFog computing has emerged as a computing paradigm for resource-restricted Internet of things (IoT) devices to support time-sensitive and computationally intensive applications. Offloading can be utilized to transfer resource-intensive tasks from resource-limited end devices to a resource-rich fog or cloud layer to reduce end-to-end latency and enhance the performance of the system. However, this advantage is still challenging to achieve in systems with a high request rate because it leads to long queues of tasks in fog nodes and reveals inefficiencies in terms of delays. In this regard, reinforcement learning (RL) is a well-known method for addressing such decision-making issues. However, in large-scale wireless networks, both action and state spaces are complex and extremely extensive. Consequently, reinforcement learning techniques may not be able to identify an efficient strategy within an acceptable time frame. Hence, deep reinforcement learning (DRL) was developed to integrate RL and deep learning (DL) to address this problem. This paper presents a systematic analysis of using RL or DRL algorithms to address offloading-related issues in fog computing. First, the taxonomy of fog computing offloading mechanisms based on RL and DRL algorithms was divided into three major categories: value-based, policy-based, and hybrid-based algorithms. These categories were then compared based on important features, including offloading problem formulation, utilized techniques, performance metrics, evaluation tools, case studies, their strengths and drawbacks, offloading directions, offloading mode, SDN-based architecture, and offloading decisions. Finally, the future research directions and open issues are discussed thoroughly.https://ieeexplore.ieee.org/document/10035389/Fog computingInternet of Things (IoT)offloadingreinforcement learningdeep reinforcement learning |
| spellingShingle | Dezheen H. Abdulazeez Shavan K. Askar Offloading Mechanisms Based on Reinforcement Learning and Deep Learning Algorithms in the Fog Computing Environment IEEE Access Fog computing Internet of Things (IoT) offloading reinforcement learning deep reinforcement learning |
| title | Offloading Mechanisms Based on Reinforcement Learning and Deep Learning Algorithms in the Fog Computing Environment |
| title_full | Offloading Mechanisms Based on Reinforcement Learning and Deep Learning Algorithms in the Fog Computing Environment |
| title_fullStr | Offloading Mechanisms Based on Reinforcement Learning and Deep Learning Algorithms in the Fog Computing Environment |
| title_full_unstemmed | Offloading Mechanisms Based on Reinforcement Learning and Deep Learning Algorithms in the Fog Computing Environment |
| title_short | Offloading Mechanisms Based on Reinforcement Learning and Deep Learning Algorithms in the Fog Computing Environment |
| title_sort | offloading mechanisms based on reinforcement learning and deep learning algorithms in the fog computing environment |
| topic | Fog computing Internet of Things (IoT) offloading reinforcement learning deep reinforcement learning |
| url | https://ieeexplore.ieee.org/document/10035389/ |
| work_keys_str_mv | AT dezheenhabdulazeez offloadingmechanismsbasedonreinforcementlearninganddeeplearningalgorithmsinthefogcomputingenvironment AT shavankaskar offloadingmechanismsbasedonreinforcementlearninganddeeplearningalgorithmsinthefogcomputingenvironment |