A NOVEL TRUE REAL-TIME SPATIOTEMPORAL DATA STREAM PROCESSING FRAMEWORK
The ability to interpret spatiotemporal data streams in real-time is critical for a range of systems. However, processing vast amounts of spatiotemporal data out of several sources, such as online traffic, social platforms, sensor networks, and other sources, is a considerable challenge. The major g...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Scientific Research Support Fund of Jordan (SRSF) and Princess Sumaya University for Technology (PSUT)
2022-09-01
|
| Series: | Jordanian Journal of Computers and Information Technology |
| Subjects: | |
| Online Access: | http://www.ejmanager.com/fulltextpdf.php?mno=99913 |
| _version_ | 1811337574974750720 |
|---|---|
| author | ATURE ANGBERA HUAH YONG CHAN |
| author_facet | ATURE ANGBERA HUAH YONG CHAN |
| author_sort | ATURE ANGBERA |
| collection | DOAJ |
| description | The ability to interpret spatiotemporal data streams in real-time is critical for a range of systems. However, processing vast amounts of spatiotemporal data out of several sources, such as online traffic, social platforms, sensor networks, and other sources, is a considerable challenge. The major goal of this study is to create a framework for processing and analyzing spatiotemporal data from multiple sources with irregular shapes so that researchers can focus on data analysis instead of worrying about the data sources' structure. We introduced a novel spatiotemporal data paradigm for true-real-time stream processing, which enables high-speed and low-latency real-time data processing, with these considerations in mind. A comparison of two state-of-the-art real-time process architectures was offered, as well as a full review of the various open-source technologies for real-time data stream processing, and their system topologies were also presented. Hence, this study proposed a brand-new framework that integrates Apache Kafka for spatiotemporal data ingestion, Apache flink for true real-time processing of spatiotemporal stream data, as well as machine learning for real-time predictions, and Apache Cassandra at the storage layer for distributed storage in real-time. The proposed framework was compared with others from the literature using the following features: Scalability (Sc), prediction tools (PT), data analytics (DA), multiple event types (MET), data storage (DS), Real-time (Rt), and performance evaluation (PE) stream processing (SP), and our proposed framework provided the ability to handle all of this task. [JJCIT 2022; 8(3.000): 256-270] |
| first_indexed | 2024-04-13T17:57:00Z |
| format | Article |
| id | doaj.art-694634e4991244908255b523a44abadf |
| institution | Directory Open Access Journal |
| issn | 2413-9351 |
| language | English |
| last_indexed | 2024-04-13T17:57:00Z |
| publishDate | 2022-09-01 |
| publisher | Scientific Research Support Fund of Jordan (SRSF) and Princess Sumaya University for Technology (PSUT) |
| record_format | Article |
| series | Jordanian Journal of Computers and Information Technology |
| spelling | doaj.art-694634e4991244908255b523a44abadf2022-12-22T02:36:26ZengScientific Research Support Fund of Jordan (SRSF) and Princess Sumaya University for Technology (PSUT)Jordanian Journal of Computers and Information Technology2413-93512022-09-018325627010.5455/jjcit.71-164683883099913A NOVEL TRUE REAL-TIME SPATIOTEMPORAL DATA STREAM PROCESSING FRAMEWORKATURE ANGBERA0HUAH YONG CHAN1Universiti Sains Malaysia, School of Computer Sciences, Pulau Pinang 11800, Malaysia Universiti Sains Malaysia, School of Computer Sciences, Pulau Pinang 11800, MalaysiaThe ability to interpret spatiotemporal data streams in real-time is critical for a range of systems. However, processing vast amounts of spatiotemporal data out of several sources, such as online traffic, social platforms, sensor networks, and other sources, is a considerable challenge. The major goal of this study is to create a framework for processing and analyzing spatiotemporal data from multiple sources with irregular shapes so that researchers can focus on data analysis instead of worrying about the data sources' structure. We introduced a novel spatiotemporal data paradigm for true-real-time stream processing, which enables high-speed and low-latency real-time data processing, with these considerations in mind. A comparison of two state-of-the-art real-time process architectures was offered, as well as a full review of the various open-source technologies for real-time data stream processing, and their system topologies were also presented. Hence, this study proposed a brand-new framework that integrates Apache Kafka for spatiotemporal data ingestion, Apache flink for true real-time processing of spatiotemporal stream data, as well as machine learning for real-time predictions, and Apache Cassandra at the storage layer for distributed storage in real-time. The proposed framework was compared with others from the literature using the following features: Scalability (Sc), prediction tools (PT), data analytics (DA), multiple event types (MET), data storage (DS), Real-time (Rt), and performance evaluation (PE) stream processing (SP), and our proposed framework provided the ability to handle all of this task. [JJCIT 2022; 8(3.000): 256-270]http://www.ejmanager.com/fulltextpdf.php?mno=99913spatiotemporal big datareal-time processingstream processingapache kafkaapache flinkapache cassandraapache spark |
| spellingShingle | ATURE ANGBERA HUAH YONG CHAN A NOVEL TRUE REAL-TIME SPATIOTEMPORAL DATA STREAM PROCESSING FRAMEWORK Jordanian Journal of Computers and Information Technology spatiotemporal big data real-time processing stream processing apache kafka apache flink apache cassandra apache spark |
| title | A NOVEL TRUE REAL-TIME SPATIOTEMPORAL DATA STREAM PROCESSING FRAMEWORK |
| title_full | A NOVEL TRUE REAL-TIME SPATIOTEMPORAL DATA STREAM PROCESSING FRAMEWORK |
| title_fullStr | A NOVEL TRUE REAL-TIME SPATIOTEMPORAL DATA STREAM PROCESSING FRAMEWORK |
| title_full_unstemmed | A NOVEL TRUE REAL-TIME SPATIOTEMPORAL DATA STREAM PROCESSING FRAMEWORK |
| title_short | A NOVEL TRUE REAL-TIME SPATIOTEMPORAL DATA STREAM PROCESSING FRAMEWORK |
| title_sort | novel true real time spatiotemporal data stream processing framework |
| topic | spatiotemporal big data real-time processing stream processing apache kafka apache flink apache cassandra apache spark |
| url | http://www.ejmanager.com/fulltextpdf.php?mno=99913 |
| work_keys_str_mv | AT atureangbera anoveltruerealtimespatiotemporaldatastreamprocessingframework AT huahyongchan anoveltruerealtimespatiotemporaldatastreamprocessingframework AT atureangbera noveltruerealtimespatiotemporaldatastreamprocessingframework AT huahyongchan noveltruerealtimespatiotemporaldatastreamprocessingframework |