Real-coded multi-objective genetic algorithm with effective queuing model for efficient job scheduling in heterogeneous Hadoop environment
In the recent past, cloud infrastructure has considerably increased its applicability. This has resulted in effective Big Data processing. Hadoop schedulers are critical components to deliver required levels of efficiency, where MapReduce tasks are assigned to Hadoop nodes by the scheduler. There is...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Elsevier
2022-06-01
|
| Series: | Journal of King Saud University: Computer and Information Sciences |
| Subjects: | |
| Online Access: | http://www.sciencedirect.com/science/article/pii/S1319157820304298 |
| _version_ | 1818240411838185472 |
|---|---|
| author | V. Seethalakshmi V Govindasamy V. Akila |
| author_facet | V. Seethalakshmi V Govindasamy V. Akila |
| author_sort | V. Seethalakshmi |
| collection | DOAJ |
| description | In the recent past, cloud infrastructure has considerably increased its applicability. This has resulted in effective Big Data processing. Hadoop schedulers are critical components to deliver required levels of efficiency, where MapReduce tasks are assigned to Hadoop nodes by the scheduler. There is a significant challenge in planning the increasing amount of functions and resources in a scalable way. Also, this challenge is further compounded by the potential heterogeneity of the deployed Hadoop. This paper proposes a scheduler that makes scheduling a choice by assessing the entire task group in the job queue. Further, the proposed scheduler uses a new scheduling method based on Real Coded Genetic Algorithm (RCGA). RCGA with MapReduce enables users to create more scalable applications. A higher abstraction is provided in less time. The experimental results indicate that the proposed RCGA scheduler achieves better performance than the existing systems for the following metrics: Execution Time, Total Cost, Resource Utilization, Speedup, Throughput, Scheduling Efficiency, Fairness Relaxation, Scheduling Time, Turnaround Time, CPU Time, Data Locality and Average Node-Locality Ratio. |
| first_indexed | 2024-12-12T13:13:01Z |
| format | Article |
| id | doaj.art-38489626631d4556981329dd012a6765 |
| institution | Directory Open Access Journal |
| issn | 1319-1578 |
| language | English |
| last_indexed | 2024-12-12T13:13:01Z |
| publishDate | 2022-06-01 |
| publisher | Elsevier |
| record_format | Article |
| series | Journal of King Saud University: Computer and Information Sciences |
| spelling | doaj.art-38489626631d4556981329dd012a67652022-12-22T00:23:28ZengElsevierJournal of King Saud University: Computer and Information Sciences1319-15782022-06-0134631783190Real-coded multi-objective genetic algorithm with effective queuing model for efficient job scheduling in heterogeneous Hadoop environmentV. Seethalakshmi0V Govindasamy1V. Akila2Department of Computer Science and Engineering, Pondicherry Engineering College Puducherry, India; Corresponding author.Department of Information Technology, Pondicherry Engineering College Puducherry, IndiaDepartment of Computer Science and Engineering, Pondicherry Engineering College Puducherry, IndiaIn the recent past, cloud infrastructure has considerably increased its applicability. This has resulted in effective Big Data processing. Hadoop schedulers are critical components to deliver required levels of efficiency, where MapReduce tasks are assigned to Hadoop nodes by the scheduler. There is a significant challenge in planning the increasing amount of functions and resources in a scalable way. Also, this challenge is further compounded by the potential heterogeneity of the deployed Hadoop. This paper proposes a scheduler that makes scheduling a choice by assessing the entire task group in the job queue. Further, the proposed scheduler uses a new scheduling method based on Real Coded Genetic Algorithm (RCGA). RCGA with MapReduce enables users to create more scalable applications. A higher abstraction is provided in less time. The experimental results indicate that the proposed RCGA scheduler achieves better performance than the existing systems for the following metrics: Execution Time, Total Cost, Resource Utilization, Speedup, Throughput, Scheduling Efficiency, Fairness Relaxation, Scheduling Time, Turnaround Time, CPU Time, Data Locality and Average Node-Locality Ratio.http://www.sciencedirect.com/science/article/pii/S1319157820304298Hadoop frameworkHeterogeneous Hadoop frameworkBig DataMapReduceGenetic algorithm |
| spellingShingle | V. Seethalakshmi V Govindasamy V. Akila Real-coded multi-objective genetic algorithm with effective queuing model for efficient job scheduling in heterogeneous Hadoop environment Journal of King Saud University: Computer and Information Sciences Hadoop framework Heterogeneous Hadoop framework Big Data MapReduce Genetic algorithm |
| title | Real-coded multi-objective genetic algorithm with effective queuing model for efficient job scheduling in heterogeneous Hadoop environment |
| title_full | Real-coded multi-objective genetic algorithm with effective queuing model for efficient job scheduling in heterogeneous Hadoop environment |
| title_fullStr | Real-coded multi-objective genetic algorithm with effective queuing model for efficient job scheduling in heterogeneous Hadoop environment |
| title_full_unstemmed | Real-coded multi-objective genetic algorithm with effective queuing model for efficient job scheduling in heterogeneous Hadoop environment |
| title_short | Real-coded multi-objective genetic algorithm with effective queuing model for efficient job scheduling in heterogeneous Hadoop environment |
| title_sort | real coded multi objective genetic algorithm with effective queuing model for efficient job scheduling in heterogeneous hadoop environment |
| topic | Hadoop framework Heterogeneous Hadoop framework Big Data MapReduce Genetic algorithm |
| url | http://www.sciencedirect.com/science/article/pii/S1319157820304298 |
| work_keys_str_mv | AT vseethalakshmi realcodedmultiobjectivegeneticalgorithmwitheffectivequeuingmodelforefficientjobschedulinginheterogeneoushadoopenvironment AT vgovindasamy realcodedmultiobjectivegeneticalgorithmwitheffectivequeuingmodelforefficientjobschedulinginheterogeneoushadoopenvironment AT vakila realcodedmultiobjectivegeneticalgorithmwitheffectivequeuingmodelforefficientjobschedulinginheterogeneoushadoopenvironment |