S-Store: streaming meets transaction processing
Stream processing addresses the needs of real-time applications. Transaction processing addresses the coordination and safety of short atomic computations. Heretofore, these two modes of operation existed in separate, stove-piped systems. In this work, we attempt to fuse the two computational paradi...
| Main Authors: | , , , , , , , , , , , , |
|---|---|
| Other Authors: | |
| Format: | Article |
| Language: | en_US |
| Published: |
Association for Computing Machinery
2018
|
| Online Access: | http://hdl.handle.net/1721.1/113832 https://orcid.org/0000-0002-7470-3265 https://orcid.org/0000-0001-9184-9058 |
| _version_ | 1811083420822929408 |
|---|---|
| author | Meehan, John Pavlo, Andrew Tufte, Kristin Zdonik, Stan Aslantas, Cansu Cetintemel, Ugur Du, Jiang Kraska, Tim Maier, David Tatbul Bitim, Emine Nesime Madden, Samuel R Stonebraker, Michael Wang, Hao |
| author2 | Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory |
| author_facet | Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory Meehan, John Pavlo, Andrew Tufte, Kristin Zdonik, Stan Aslantas, Cansu Cetintemel, Ugur Du, Jiang Kraska, Tim Maier, David Tatbul Bitim, Emine Nesime Madden, Samuel R Stonebraker, Michael Wang, Hao |
| author_sort | Meehan, John |
| collection | MIT |
| description | Stream processing addresses the needs of real-time applications. Transaction processing addresses the coordination and safety of short atomic computations. Heretofore, these two modes of operation existed in separate, stove-piped systems. In this work, we attempt to fuse the two computational paradigms in a single system called S-Store. In this way, S-Store can simultaneously accommodate OLTP and streaming applications. We present a simple transaction model for streams that integrates seamlessly with a traditional OLTP system, and provides both ACID and stream-oriented guarantees. We chose to build S-Store as an extension of H-Store - an open-source, in-memory, distributed OLTP database system. By implementing S-Store in this way, we can make use of the transaction processing facilities that H-Store already provides, and we can concentrate on the additional features that are needed to support streaming. Similar implementations could be done using other main-memory OLTP platforms. We show that we can actually achieve higher throughput for streaming workloads in S-Store than an equivalent deployment in H-Store alone. We also show how this can be achieved within H-Store with the addition of a modest amount of new functionality. Furthermore, we compare S-Store to two state-of-the-art streaming systems, Esper and Apache Storm, and show how S-Store can sometimes exceed their performance while at the same time providing stronger correctness guarantees. |
| first_indexed | 2024-09-23T12:32:47Z |
| format | Article |
| id | mit-1721.1/113832 |
| institution | Massachusetts Institute of Technology |
| language | en_US |
| last_indexed | 2024-09-23T12:32:47Z |
| publishDate | 2018 |
| publisher | Association for Computing Machinery |
| record_format | dspace |
| spelling | mit-1721.1/1138322022-09-28T08:31:22Z S-Store: streaming meets transaction processing Meehan, John Pavlo, Andrew Tufte, Kristin Zdonik, Stan Aslantas, Cansu Cetintemel, Ugur Du, Jiang Kraska, Tim Maier, David Tatbul Bitim, Emine Nesime Madden, Samuel R Stonebraker, Michael Wang, Hao Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science Tatbul Bitim, Emine Nesime Madden, Samuel R Stonebraker, Michael Wang, Hao Stream processing addresses the needs of real-time applications. Transaction processing addresses the coordination and safety of short atomic computations. Heretofore, these two modes of operation existed in separate, stove-piped systems. In this work, we attempt to fuse the two computational paradigms in a single system called S-Store. In this way, S-Store can simultaneously accommodate OLTP and streaming applications. We present a simple transaction model for streams that integrates seamlessly with a traditional OLTP system, and provides both ACID and stream-oriented guarantees. We chose to build S-Store as an extension of H-Store - an open-source, in-memory, distributed OLTP database system. By implementing S-Store in this way, we can make use of the transaction processing facilities that H-Store already provides, and we can concentrate on the additional features that are needed to support streaming. Similar implementations could be done using other main-memory OLTP platforms. We show that we can actually achieve higher throughput for streaming workloads in S-Store than an equivalent deployment in H-Store alone. We also show how this can be achieved within H-Store with the addition of a modest amount of new functionality. Furthermore, we compare S-Store to two state-of-the-art streaming systems, Esper and Apache Storm, and show how S-Store can sometimes exceed their performance while at the same time providing stronger correctness guarantees. Intel Science and Technology Center for Big Data National Science Foundation (U.S.) (IIS-1111423) National Science Foundation (U.S.) (IIS-1110917) 2018-02-20T15:51:28Z 2018-02-20T15:51:28Z 2015-09 Article http://purl.org/eprint/type/ConferencePaper 2150-8097 http://hdl.handle.net/1721.1/113832 Meehan, John, et al. “S-Store: Streaming Meets Transaction Processing.” Proceedings of the VLDB Endowment, vol. 8, no. 13, Sept. 2015, pp. 2134–45. https://orcid.org/0000-0002-7470-3265 https://orcid.org/0000-0001-9184-9058 en_US http://dx.doi.org/10.14778/2831360.2831367 Proceedings of the VLDB Endowment Creative Commons Attribution-NonCommercial-NoDerivs License http://creativecommons.org/licenses/by-nc-nd/3.0/ application/pdf Association for Computing Machinery ACM |
| spellingShingle | Meehan, John Pavlo, Andrew Tufte, Kristin Zdonik, Stan Aslantas, Cansu Cetintemel, Ugur Du, Jiang Kraska, Tim Maier, David Tatbul Bitim, Emine Nesime Madden, Samuel R Stonebraker, Michael Wang, Hao S-Store: streaming meets transaction processing |
| title | S-Store: streaming meets transaction processing |
| title_full | S-Store: streaming meets transaction processing |
| title_fullStr | S-Store: streaming meets transaction processing |
| title_full_unstemmed | S-Store: streaming meets transaction processing |
| title_short | S-Store: streaming meets transaction processing |
| title_sort | s store streaming meets transaction processing |
| url | http://hdl.handle.net/1721.1/113832 https://orcid.org/0000-0002-7470-3265 https://orcid.org/0000-0001-9184-9058 |
| work_keys_str_mv | AT meehanjohn sstorestreamingmeetstransactionprocessing AT pavloandrew sstorestreamingmeetstransactionprocessing AT tuftekristin sstorestreamingmeetstransactionprocessing AT zdonikstan sstorestreamingmeetstransactionprocessing AT aslantascansu sstorestreamingmeetstransactionprocessing AT cetintemelugur sstorestreamingmeetstransactionprocessing AT dujiang sstorestreamingmeetstransactionprocessing AT kraskatim sstorestreamingmeetstransactionprocessing AT maierdavid sstorestreamingmeetstransactionprocessing AT tatbulbitimeminenesime sstorestreamingmeetstransactionprocessing AT maddensamuelr sstorestreamingmeetstransactionprocessing AT stonebrakermichael sstorestreamingmeetstransactionprocessing AT wanghao sstorestreamingmeetstransactionprocessing |