Pipelined Dynamic Scheduling of Big Data Streams
We are currently living in the big data era, in which it has become more necessary than ever to develop “smart” schedulers. It is common knowledge that the default Storm scheduler, as well as a large number of static schemes, has presented certain deficiencies. One of the most important of these def...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2020-07-01
|
| Series: | Applied Sciences |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2076-3417/10/14/4796 |
| _version_ | 1797562623178309632 |
|---|---|
| author | Stavros Souravlas Sofia Anastasiadou |
| author_facet | Stavros Souravlas Sofia Anastasiadou |
| author_sort | Stavros Souravlas |
| collection | DOAJ |
| description | We are currently living in the big data era, in which it has become more necessary than ever to develop “smart” schedulers. It is common knowledge that the default Storm scheduler, as well as a large number of static schemes, has presented certain deficiencies. One of the most important of these deficiencies is the weakness in handling cases in which system changes occur. In such a scenario, some type of re-scheduling is necessary to keep the system working in the most efficient way. In this paper, we present a pipeline-based dynamic modular arithmetic-based scheduler (PMOD scheduler), which can be used to re-schedule the streams distributed among a set of nodes and their tasks, when the system parameters (number of tasks, executors or nodes) change. The PMOD scheduler organizes all the required operations in a pipeline scheme, thus reducing the overall processing time. |
| first_indexed | 2024-03-10T18:31:34Z |
| format | Article |
| id | doaj.art-71dcb684a79346a69e7037acc56e41c3 |
| institution | Directory Open Access Journal |
| issn | 2076-3417 |
| language | English |
| last_indexed | 2024-03-10T18:31:34Z |
| publishDate | 2020-07-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Applied Sciences |
| spelling | doaj.art-71dcb684a79346a69e7037acc56e41c32023-11-20T06:36:22ZengMDPI AGApplied Sciences2076-34172020-07-011014479610.3390/app10144796Pipelined Dynamic Scheduling of Big Data StreamsStavros Souravlas0Sofia Anastasiadou1Department of Applied Informatics, School of Information Sciences, University of Macedonia Thessaloniki, 54616 Thessaloniki, GreeceDepartment of Early Childhood Education, Faculty of Education, University of Western Macedonia, 21, 53100 Florina, GreeceWe are currently living in the big data era, in which it has become more necessary than ever to develop “smart” schedulers. It is common knowledge that the default Storm scheduler, as well as a large number of static schemes, has presented certain deficiencies. One of the most important of these deficiencies is the weakness in handling cases in which system changes occur. In such a scenario, some type of re-scheduling is necessary to keep the system working in the most efficient way. In this paper, we present a pipeline-based dynamic modular arithmetic-based scheduler (PMOD scheduler), which can be used to re-schedule the streams distributed among a set of nodes and their tasks, when the system parameters (number of tasks, executors or nodes) change. The PMOD scheduler organizes all the required operations in a pipeline scheme, thus reducing the overall processing time.https://www.mdpi.com/2076-3417/10/14/4796cloud computingbig datatask re-schedulingtask distributionarchitecturepipeline |
| spellingShingle | Stavros Souravlas Sofia Anastasiadou Pipelined Dynamic Scheduling of Big Data Streams Applied Sciences cloud computing big data task re-scheduling task distribution architecture pipeline |
| title | Pipelined Dynamic Scheduling of Big Data Streams |
| title_full | Pipelined Dynamic Scheduling of Big Data Streams |
| title_fullStr | Pipelined Dynamic Scheduling of Big Data Streams |
| title_full_unstemmed | Pipelined Dynamic Scheduling of Big Data Streams |
| title_short | Pipelined Dynamic Scheduling of Big Data Streams |
| title_sort | pipelined dynamic scheduling of big data streams |
| topic | cloud computing big data task re-scheduling task distribution architecture pipeline |
| url | https://www.mdpi.com/2076-3417/10/14/4796 |
| work_keys_str_mv | AT stavrossouravlas pipelineddynamicschedulingofbigdatastreams AT sofiaanastasiadou pipelineddynamicschedulingofbigdatastreams |