Real-Time Analytics: Concepts, Architectures, and ML/AI Considerations
With the advancement in intelligent devices, social media, and the Internet of Things, staggering amounts of new data are being generated, and the pace is continuously accelerating. Real-time analytics (RTA) has emerged as a distinct branch of big data analytics focusing on the velocity aspect of bi...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2023-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10183999/ |
| _version_ | 1797737428110278656 |
|---|---|
| author | Weisi Chen Zoran Milosevic Fethi A. Rabhi Andrew Berry |
| author_facet | Weisi Chen Zoran Milosevic Fethi A. Rabhi Andrew Berry |
| author_sort | Weisi Chen |
| collection | DOAJ |
| description | With the advancement in intelligent devices, social media, and the Internet of Things, staggering amounts of new data are being generated, and the pace is continuously accelerating. Real-time analytics (RTA) has emerged as a distinct branch of big data analytics focusing on the velocity aspect of big data, in which data is prepared, processed, and analyzed as it arrives, intending to generate insights and create business value in near real-time. The objective of this paper is to provide an overview of key concepts and architectural approaches for designing RTA solutions, including the relevant infrastructure, processing, and analytics platforms, as well as analytics techniques and tools with the most up-to-date machine learning and artificial intelligence considerations, and position these in the context of the most prominent platforms and analytics techniques. The paper develops a logical analytics stack to support the description of key functionality and relationships between relevant components in RTA solutions based on a thorough literature review and industrial practice. This provides practitioners with guidance in selecting the most appropriate solutions for their RTA problems, including the application of emerging AI technologies in this context. The paper discusses the complex event processing technology that has influenced many recent data streaming solutions in the analytics stack and highlights the integration of machine learning and artificial intelligence into RTA solutions. Some real-life application scenarios in the finance and health domains are presented, including several of the authors’ earlier contributions, to demonstrate the utilization of the techniques and technologies discussed in this paper. Future research directions and remaining challenges are discussed. |
| first_indexed | 2024-03-12T13:28:34Z |
| format | Article |
| id | doaj.art-877978435b0046fe88819972c5e6d472 |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-03-12T13:28:34Z |
| publishDate | 2023-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-877978435b0046fe88819972c5e6d4722023-08-24T23:00:16ZengIEEEIEEE Access2169-35362023-01-0111716347165710.1109/ACCESS.2023.329569410183999Real-Time Analytics: Concepts, Architectures, and ML/AI ConsiderationsWeisi Chen0https://orcid.org/0000-0001-8131-392XZoran Milosevic1https://orcid.org/0000-0002-1364-7423Fethi A. Rabhi2https://orcid.org/0000-0001-8934-6259Andrew Berry3School of Software Engineering, Xiamen University of Technology, Jimei, Xiamen, ChinaDeontik, Brisbane, QLD, AustraliaSchool of Computer Science and Engineering, University of New South Wales, Sydney, NSW, AustraliaDeontik, Brisbane, QLD, AustraliaWith the advancement in intelligent devices, social media, and the Internet of Things, staggering amounts of new data are being generated, and the pace is continuously accelerating. Real-time analytics (RTA) has emerged as a distinct branch of big data analytics focusing on the velocity aspect of big data, in which data is prepared, processed, and analyzed as it arrives, intending to generate insights and create business value in near real-time. The objective of this paper is to provide an overview of key concepts and architectural approaches for designing RTA solutions, including the relevant infrastructure, processing, and analytics platforms, as well as analytics techniques and tools with the most up-to-date machine learning and artificial intelligence considerations, and position these in the context of the most prominent platforms and analytics techniques. The paper develops a logical analytics stack to support the description of key functionality and relationships between relevant components in RTA solutions based on a thorough literature review and industrial practice. This provides practitioners with guidance in selecting the most appropriate solutions for their RTA problems, including the application of emerging AI technologies in this context. The paper discusses the complex event processing technology that has influenced many recent data streaming solutions in the analytics stack and highlights the integration of machine learning and artificial intelligence into RTA solutions. Some real-life application scenarios in the finance and health domains are presented, including several of the authors’ earlier contributions, to demonstrate the utilization of the techniques and technologies discussed in this paper. Future research directions and remaining challenges are discussed.https://ieeexplore.ieee.org/document/10183999/Real-time analyticsdata streamingbig data analyticscomplex event processingmachine learning |
| spellingShingle | Weisi Chen Zoran Milosevic Fethi A. Rabhi Andrew Berry Real-Time Analytics: Concepts, Architectures, and ML/AI Considerations IEEE Access Real-time analytics data streaming big data analytics complex event processing machine learning |
| title | Real-Time Analytics: Concepts, Architectures, and ML/AI Considerations |
| title_full | Real-Time Analytics: Concepts, Architectures, and ML/AI Considerations |
| title_fullStr | Real-Time Analytics: Concepts, Architectures, and ML/AI Considerations |
| title_full_unstemmed | Real-Time Analytics: Concepts, Architectures, and ML/AI Considerations |
| title_short | Real-Time Analytics: Concepts, Architectures, and ML/AI Considerations |
| title_sort | real time analytics concepts architectures and ml ai considerations |
| topic | Real-time analytics data streaming big data analytics complex event processing machine learning |
| url | https://ieeexplore.ieee.org/document/10183999/ |
| work_keys_str_mv | AT weisichen realtimeanalyticsconceptsarchitecturesandmlaiconsiderations AT zoranmilosevic realtimeanalyticsconceptsarchitecturesandmlaiconsiderations AT fethiarabhi realtimeanalyticsconceptsarchitecturesandmlaiconsiderations AT andrewberry realtimeanalyticsconceptsarchitecturesandmlaiconsiderations |