A Modular Framework for Data Processing at the Edge: Design and Implementation
There is a rapid increase in the number of edge devices in IoT solutions, generating vast amounts of data that need to be processed and analyzed efficiently. Traditional cloud-based architectures can face latency, bandwidth, and privacy challenges when dealing with this data flood. There is currentl...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-09-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/23/17/7662 |
| _version_ | 1827727731096289280 |
|---|---|
| author | Lubomir Urblik Erik Kajati Peter Papcun Iveta Zolotova |
| author_facet | Lubomir Urblik Erik Kajati Peter Papcun Iveta Zolotova |
| author_sort | Lubomir Urblik |
| collection | DOAJ |
| description | There is a rapid increase in the number of edge devices in IoT solutions, generating vast amounts of data that need to be processed and analyzed efficiently. Traditional cloud-based architectures can face latency, bandwidth, and privacy challenges when dealing with this data flood. There is currently no unified approach to the creation of edge computing solutions. This work addresses this problem by exploring containerization for data processing solutions at the network’s edge. The current approach involves creating a specialized application compatible with the device used. Another approach involves using containerization for deployment and monitoring. The heterogeneity of edge environments would greatly benefit from a universal modular platform. Our proposed edge computing-based framework implements a streaming extract, transform, and load pipeline for data processing and analysis using ZeroMQ as the communication backbone and containerization for scalable deployment. Results demonstrate the effectiveness of the proposed framework, making it suitable for time-sensitive IoT applications. |
| first_indexed | 2024-03-10T23:12:10Z |
| format | Article |
| id | doaj.art-32acc195375d478c9d387911b31e03d1 |
| institution | Directory Open Access Journal |
| issn | 1424-8220 |
| language | English |
| last_indexed | 2024-03-10T23:12:10Z |
| publishDate | 2023-09-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj.art-32acc195375d478c9d387911b31e03d12023-11-19T08:52:52ZengMDPI AGSensors1424-82202023-09-012317766210.3390/s23177662A Modular Framework for Data Processing at the Edge: Design and ImplementationLubomir Urblik0Erik Kajati1Peter Papcun2Iveta Zolotova3Department of Cybernetics and Artificial Intelligence, Faculty of EE & Informatics, Technical University of Kosice, 042 00 Kosice, SlovakiaDepartment of Cybernetics and Artificial Intelligence, Faculty of EE & Informatics, Technical University of Kosice, 042 00 Kosice, SlovakiaDepartment of Cybernetics and Artificial Intelligence, Faculty of EE & Informatics, Technical University of Kosice, 042 00 Kosice, SlovakiaDepartment of Cybernetics and Artificial Intelligence, Faculty of EE & Informatics, Technical University of Kosice, 042 00 Kosice, SlovakiaThere is a rapid increase in the number of edge devices in IoT solutions, generating vast amounts of data that need to be processed and analyzed efficiently. Traditional cloud-based architectures can face latency, bandwidth, and privacy challenges when dealing with this data flood. There is currently no unified approach to the creation of edge computing solutions. This work addresses this problem by exploring containerization for data processing solutions at the network’s edge. The current approach involves creating a specialized application compatible with the device used. Another approach involves using containerization for deployment and monitoring. The heterogeneity of edge environments would greatly benefit from a universal modular platform. Our proposed edge computing-based framework implements a streaming extract, transform, and load pipeline for data processing and analysis using ZeroMQ as the communication backbone and containerization for scalable deployment. Results demonstrate the effectiveness of the proposed framework, making it suitable for time-sensitive IoT applications.https://www.mdpi.com/1424-8220/23/17/7662containerizationedge computingdata processing frameworkKubernetesDocker |
| spellingShingle | Lubomir Urblik Erik Kajati Peter Papcun Iveta Zolotova A Modular Framework for Data Processing at the Edge: Design and Implementation Sensors containerization edge computing data processing framework Kubernetes Docker |
| title | A Modular Framework for Data Processing at the Edge: Design and Implementation |
| title_full | A Modular Framework for Data Processing at the Edge: Design and Implementation |
| title_fullStr | A Modular Framework for Data Processing at the Edge: Design and Implementation |
| title_full_unstemmed | A Modular Framework for Data Processing at the Edge: Design and Implementation |
| title_short | A Modular Framework for Data Processing at the Edge: Design and Implementation |
| title_sort | modular framework for data processing at the edge design and implementation |
| topic | containerization edge computing data processing framework Kubernetes Docker |
| url | https://www.mdpi.com/1424-8220/23/17/7662 |
| work_keys_str_mv | AT lubomirurblik amodularframeworkfordataprocessingattheedgedesignandimplementation AT erikkajati amodularframeworkfordataprocessingattheedgedesignandimplementation AT peterpapcun amodularframeworkfordataprocessingattheedgedesignandimplementation AT ivetazolotova amodularframeworkfordataprocessingattheedgedesignandimplementation AT lubomirurblik modularframeworkfordataprocessingattheedgedesignandimplementation AT erikkajati modularframeworkfordataprocessingattheedgedesignandimplementation AT peterpapcun modularframeworkfordataprocessingattheedgedesignandimplementation AT ivetazolotova modularframeworkfordataprocessingattheedgedesignandimplementation |