DESIGN OF COMPREHENSIVE FRAMEWORK ON OPTIMIZATION METHODS IN DISTRIBUTED CLUSTERS
MapReduce is a popular, open source programming paradigm to handle big data which is an industry standard large scale data processing system used by many companies like Yahoo, Google, Facebook, etc. The YARN framework uses low resource fairness algorithms such as FIFO, Capacity, Fair, DRF scheduler...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
European Alliance for Innovation (EAI)
2018-09-01
|
| Series: | EAI Endorsed Transactions on Energy Web |
| Subjects: | |
| Online Access: | https://publications.eai.eu/index.php/ew/article/view/967 |
| _version_ | 1797989592816680960 |
|---|---|
| author | Dr. Kiran Kumar Pulamolu Dr. D. Venkata Subramanian Dr Krishnaraj |
| author_facet | Dr. Kiran Kumar Pulamolu Dr. D. Venkata Subramanian Dr Krishnaraj |
| author_sort | Dr. Kiran Kumar Pulamolu |
| collection | DOAJ |
| description |
MapReduce is a popular, open source programming paradigm to handle big data which is an industry standard large scale data processing system used by many companies like Yahoo, Google, Facebook, etc. The YARN framework uses low resource fairness algorithms such as FIFO, Capacity, Fair, DRF scheduler, whereas these schedulers are not suitable for heterogeneous Hadoop clusters. Therefore, an Enhanced Combined Regression Ranking (eCRRYARN) algorithm was proposed to enhance resource fairness. The proposed algorithm uses linear regression model to estimate the expected resources to be availed by the tenants. The order ranking is given to the estimated resource and the resources shared as per the ranking provided. Hence, the Hierarchical Hadoop Cluster Resource Sharing (HHCRS) algorithm has been adopted for hierarchical distributed cluster aiming to design a cost effective cluster for organization which is spread across the globe.
|
| first_indexed | 2024-04-11T08:21:47Z |
| format | Article |
| id | doaj.art-e08f2d10d74f441ca18be032e4615fce |
| institution | Directory Open Access Journal |
| issn | 2032-944X |
| language | English |
| last_indexed | 2024-04-11T08:21:47Z |
| publishDate | 2018-09-01 |
| publisher | European Alliance for Innovation (EAI) |
| record_format | Article |
| series | EAI Endorsed Transactions on Energy Web |
| spelling | doaj.art-e08f2d10d74f441ca18be032e4615fce2022-12-22T04:34:55ZengEuropean Alliance for Innovation (EAI)EAI Endorsed Transactions on Energy Web2032-944X2018-09-0152010.4108/eai.12-9-2018.155745DESIGN OF COMPREHENSIVE FRAMEWORK ON OPTIMIZATION METHODS IN DISTRIBUTED CLUSTERSDr. Kiran Kumar Pulamolu0Dr. D. Venkata Subramanian1Dr Krishnaraj2Sasi Institute of Technology and EngineeringHindustan Institute of Technology and Science Sasi Institute of Technology and Engineering MapReduce is a popular, open source programming paradigm to handle big data which is an industry standard large scale data processing system used by many companies like Yahoo, Google, Facebook, etc. The YARN framework uses low resource fairness algorithms such as FIFO, Capacity, Fair, DRF scheduler, whereas these schedulers are not suitable for heterogeneous Hadoop clusters. Therefore, an Enhanced Combined Regression Ranking (eCRRYARN) algorithm was proposed to enhance resource fairness. The proposed algorithm uses linear regression model to estimate the expected resources to be availed by the tenants. The order ranking is given to the estimated resource and the resources shared as per the ranking provided. Hence, the Hierarchical Hadoop Cluster Resource Sharing (HHCRS) algorithm has been adopted for hierarchical distributed cluster aiming to design a cost effective cluster for organization which is spread across the globe. https://publications.eai.eu/index.php/ew/article/view/967Distributed ClusterResource FairnessResource SharingHierarchical ClusterMapReduce |
| spellingShingle | Dr. Kiran Kumar Pulamolu Dr. D. Venkata Subramanian Dr Krishnaraj DESIGN OF COMPREHENSIVE FRAMEWORK ON OPTIMIZATION METHODS IN DISTRIBUTED CLUSTERS EAI Endorsed Transactions on Energy Web Distributed Cluster Resource Fairness Resource Sharing Hierarchical Cluster MapReduce |
| title | DESIGN OF COMPREHENSIVE FRAMEWORK ON OPTIMIZATION METHODS IN DISTRIBUTED CLUSTERS |
| title_full | DESIGN OF COMPREHENSIVE FRAMEWORK ON OPTIMIZATION METHODS IN DISTRIBUTED CLUSTERS |
| title_fullStr | DESIGN OF COMPREHENSIVE FRAMEWORK ON OPTIMIZATION METHODS IN DISTRIBUTED CLUSTERS |
| title_full_unstemmed | DESIGN OF COMPREHENSIVE FRAMEWORK ON OPTIMIZATION METHODS IN DISTRIBUTED CLUSTERS |
| title_short | DESIGN OF COMPREHENSIVE FRAMEWORK ON OPTIMIZATION METHODS IN DISTRIBUTED CLUSTERS |
| title_sort | design of comprehensive framework on optimization methods in distributed clusters |
| topic | Distributed Cluster Resource Fairness Resource Sharing Hierarchical Cluster MapReduce |
| url | https://publications.eai.eu/index.php/ew/article/view/967 |
| work_keys_str_mv | AT drkirankumarpulamolu designofcomprehensiveframeworkonoptimizationmethodsindistributedclusters AT drdvenkatasubramanian designofcomprehensiveframeworkonoptimizationmethodsindistributedclusters AT drkrishnaraj designofcomprehensiveframeworkonoptimizationmethodsindistributedclusters |