Split computing: DNN inference partition with load balancing in IoT-edge platform for beyond 5G
In the era of beyond 5G technology, it is expected that more and more applications can use deep neural network (DNN) models for different purposes with minimum inference time which is highly important for providing a great user experience in the Internet of Things (IoT) use-cases. However, due to ha...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Elsevier
2022-10-01
|
| Series: | Measurement: Sensors |
| Subjects: | |
| Online Access: | http://www.sciencedirect.com/science/article/pii/S2665917422000435 |
| _version_ | 1817999694158102528 |
|---|---|
| author | Jyotirmoy Karjee Praveen Naik S Kartik Anand Vanamala N. Bhargav |
| author_facet | Jyotirmoy Karjee Praveen Naik S Kartik Anand Vanamala N. Bhargav |
| author_sort | Jyotirmoy Karjee |
| collection | DOAJ |
| description | In the era of beyond 5G technology, it is expected that more and more applications can use deep neural network (DNN) models for different purposes with minimum inference time which is highly important for providing a great user experience in the Internet of Things (IoT) use-cases. However, due to hardware limitations and low computational capabilities, it is almost impossible to execute complex DNN models on IoT devices. To provide a better user experience and mitigate these problems, we introduce split computing technology where the DNN model inference task is partially offloaded from an IoT device to a nearby dependable device called edge. While splitting the DNN between IoT-edge, it is unclear how much the DNN model needs to be partitioned according to varying network conditions to ensure a satisfactory user experience. To address these issues, we propose the following mechanisms that provide a trade-off between the computation and communication, namely: (1) dynamic split computation (DSC) mechanism: finds an optimal partitioning of the DNN model to be executed between IoT and edge to reduce the computation overhead (i.e., overall inference time); (2) reliable communication network switching (RCNS) mechanism: intelligently finds a suitable network to connect to either Wi-Fi/Cellular/Bluetooth to provide reliable data transport between IoT and edge. Further, as the edge devices perform DNN tasks extensively, it is possible that they run out of battery, or get interrupted by high-priority user applications executing on them due to which edge devices cannot complete the assigned DNN task in the desired time. To address these issues, we also propose a load-balancing mechanism between IoT and edge called task load balancing with prioritization (TLBP) model for single and multiple DNN task scenarios. We conduct extensive experiments with Raspberry Pi as an IoT device and Samsung Galaxy S20 smartphones as edge devices to test the proposed mechanism. The results show that the proposed algorithms substantially reduce the inference time of DNN models as compared to the on-device inference time and balance the tasks across the edge devices to minimize excessive battery drainage. |
| first_indexed | 2024-04-14T03:12:27Z |
| format | Article |
| id | doaj.art-61911fd5ec604aefb95ca55d6cebcde9 |
| institution | Directory Open Access Journal |
| issn | 2665-9174 |
| language | English |
| last_indexed | 2024-04-14T03:12:27Z |
| publishDate | 2022-10-01 |
| publisher | Elsevier |
| record_format | Article |
| series | Measurement: Sensors |
| spelling | doaj.art-61911fd5ec604aefb95ca55d6cebcde92022-12-22T02:15:34ZengElsevierMeasurement: Sensors2665-91742022-10-0123100409Split computing: DNN inference partition with load balancing in IoT-edge platform for beyond 5GJyotirmoy Karjee0Praveen Naik S1Kartik Anand2Vanamala N. Bhargav3Corresponding author.; Samsung R&D Institute India, Bangalore, IndiaSamsung R&D Institute India, Bangalore, IndiaSamsung R&D Institute India, Bangalore, IndiaSamsung R&D Institute India, Bangalore, IndiaIn the era of beyond 5G technology, it is expected that more and more applications can use deep neural network (DNN) models for different purposes with minimum inference time which is highly important for providing a great user experience in the Internet of Things (IoT) use-cases. However, due to hardware limitations and low computational capabilities, it is almost impossible to execute complex DNN models on IoT devices. To provide a better user experience and mitigate these problems, we introduce split computing technology where the DNN model inference task is partially offloaded from an IoT device to a nearby dependable device called edge. While splitting the DNN between IoT-edge, it is unclear how much the DNN model needs to be partitioned according to varying network conditions to ensure a satisfactory user experience. To address these issues, we propose the following mechanisms that provide a trade-off between the computation and communication, namely: (1) dynamic split computation (DSC) mechanism: finds an optimal partitioning of the DNN model to be executed between IoT and edge to reduce the computation overhead (i.e., overall inference time); (2) reliable communication network switching (RCNS) mechanism: intelligently finds a suitable network to connect to either Wi-Fi/Cellular/Bluetooth to provide reliable data transport between IoT and edge. Further, as the edge devices perform DNN tasks extensively, it is possible that they run out of battery, or get interrupted by high-priority user applications executing on them due to which edge devices cannot complete the assigned DNN task in the desired time. To address these issues, we also propose a load-balancing mechanism between IoT and edge called task load balancing with prioritization (TLBP) model for single and multiple DNN task scenarios. We conduct extensive experiments with Raspberry Pi as an IoT device and Samsung Galaxy S20 smartphones as edge devices to test the proposed mechanism. The results show that the proposed algorithms substantially reduce the inference time of DNN models as compared to the on-device inference time and balance the tasks across the edge devices to minimize excessive battery drainage.http://www.sciencedirect.com/science/article/pii/S2665917422000435Connection reliabilityDeep neural network (DNN)Inference timeEdge computingInternet of things (IoT)Load-balancing |
| spellingShingle | Jyotirmoy Karjee Praveen Naik S Kartik Anand Vanamala N. Bhargav Split computing: DNN inference partition with load balancing in IoT-edge platform for beyond 5G Measurement: Sensors Connection reliability Deep neural network (DNN) Inference time Edge computing Internet of things (IoT) Load-balancing |
| title | Split computing: DNN inference partition with load balancing in IoT-edge platform for beyond 5G |
| title_full | Split computing: DNN inference partition with load balancing in IoT-edge platform for beyond 5G |
| title_fullStr | Split computing: DNN inference partition with load balancing in IoT-edge platform for beyond 5G |
| title_full_unstemmed | Split computing: DNN inference partition with load balancing in IoT-edge platform for beyond 5G |
| title_short | Split computing: DNN inference partition with load balancing in IoT-edge platform for beyond 5G |
| title_sort | split computing dnn inference partition with load balancing in iot edge platform for beyond 5g |
| topic | Connection reliability Deep neural network (DNN) Inference time Edge computing Internet of things (IoT) Load-balancing |
| url | http://www.sciencedirect.com/science/article/pii/S2665917422000435 |
| work_keys_str_mv | AT jyotirmoykarjee splitcomputingdnninferencepartitionwithloadbalancinginiotedgeplatformforbeyond5g AT praveennaiks splitcomputingdnninferencepartitionwithloadbalancinginiotedgeplatformforbeyond5g AT kartikanand splitcomputingdnninferencepartitionwithloadbalancinginiotedgeplatformforbeyond5g AT vanamalanbhargav splitcomputingdnninferencepartitionwithloadbalancinginiotedgeplatformforbeyond5g |