Enabling deep analytics in stream processing systems

Real-time applications often analyze data coming from sensor networks using relational and domain-specific operations such as signal processing and machine learning algorithms. To support such increasingly important scenarios, many data management systems integrate with numerical frameworks like R....

Full description

Bibliographic Details
Main Authors: Nikolic, M, Chandramouli, B, Goldstein, J
Format: Conference item
Published: Springer Verlag 2017
Description
Summary:Real-time applications often analyze data coming from sensor networks using relational and domain-specific operations such as signal processing and machine learning algorithms. To support such increasingly important scenarios, many data management systems integrate with numerical frameworks like R. Such solutions, however, incur significant performance penalties as relational engines and numerical tools operate on fundamentally different data models with expensive inter-communication mechanisms. In addition, none of these solutions supports efficient real-time and incremental analysis. In this work, we advocate a deep integration of domain-specific operations into generalpurpose query processors with the goal of providing unified query and data models for both online and offline processing. Our proof-of-concept system tightly integrates relational and digital signal processing operations and achieves orders of magnitude better performance than existing loosely-coupled data management systems.