Load balancing and service discovery using Docker Swarm for microservice based big data applications
Abstract Big Data applications require extensive resources and environments to store, process and analyze this colossal collection of data in a distributed manner. Containerization with cloud computing provides a pertinent remedy to accommodate big data requirements, however requires a precise and a...
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
SpringerOpen
2023-01-01
|
| Series: | Journal of Cloud Computing: Advances, Systems and Applications |
| Subjects: | |
| Online Access: | https://doi.org/10.1186/s13677-022-00358-7 |
| _version_ | 1797958490316079104 |
|---|---|
| author | Neelam Singh Yasir Hamid Sapna Juneja Gautam Srivastava Gaurav Dhiman Thippa Reddy Gadekallu Mohd Asif Shah |
| author_facet | Neelam Singh Yasir Hamid Sapna Juneja Gautam Srivastava Gaurav Dhiman Thippa Reddy Gadekallu Mohd Asif Shah |
| author_sort | Neelam Singh |
| collection | DOAJ |
| description | Abstract Big Data applications require extensive resources and environments to store, process and analyze this colossal collection of data in a distributed manner. Containerization with cloud computing provides a pertinent remedy to accommodate big data requirements, however requires a precise and appropriate load-balancing mechanism. The load on servers increases exponentially with increased resource usage thus making load balancing an essential requirement. Moreover, the adjustment of containers accurately and rapidly according to load as per services is one of the crucial aspects in big data applications. This study provides a review relating to containerized environments like Docker for big data applications with load balancing. A novel scheduling mechanism of containers for big data applications established on Docker Swarm and Microservice architecture is proposed. The concept of Docker Swarm is utilized to effectively handle big data applications' workload and service discovery. Results shows that increasing workloads with respect to big data applications can be effectively managed by utilizing microservices in containerized environments and load balancing is efficiently achieved using Docker Swarm. The implementation is done using a case study deployed on a single server and then scaled to four instances. Applications developed using containerized microservices reduces average deployment time and continuous integration. |
| first_indexed | 2024-04-11T00:19:50Z |
| format | Article |
| id | doaj.art-8b95e14b152141dc9bc3524c153ba4a8 |
| institution | Directory Open Access Journal |
| issn | 2192-113X |
| language | English |
| last_indexed | 2024-04-11T00:19:50Z |
| publishDate | 2023-01-01 |
| publisher | SpringerOpen |
| record_format | Article |
| series | Journal of Cloud Computing: Advances, Systems and Applications |
| spelling | doaj.art-8b95e14b152141dc9bc3524c153ba4a82023-01-08T12:20:53ZengSpringerOpenJournal of Cloud Computing: Advances, Systems and Applications2192-113X2023-01-011211910.1186/s13677-022-00358-7Load balancing and service discovery using Docker Swarm for microservice based big data applicationsNeelam Singh0Yasir Hamid1Sapna Juneja2Gautam Srivastava3Gaurav Dhiman4Thippa Reddy Gadekallu5Mohd Asif Shah6Department of Computer Science and Engineering, Graphic Era Deemed to be UniversityAbu Dhabi Polytechnic, Abu Dhabi PolytechnicKIET Group of Institutions, Delhi NCRDepartment of Mathematics and Computer Science, Brandon UniversityDepartment of Computer Science and Engineering, Graphic Era Deemed to be UniversityDepartment of Electrical and Computer Engineering, Lebanese American UniversityDepartment of Economics, Kebri Dehar UniversityAbstract Big Data applications require extensive resources and environments to store, process and analyze this colossal collection of data in a distributed manner. Containerization with cloud computing provides a pertinent remedy to accommodate big data requirements, however requires a precise and appropriate load-balancing mechanism. The load on servers increases exponentially with increased resource usage thus making load balancing an essential requirement. Moreover, the adjustment of containers accurately and rapidly according to load as per services is one of the crucial aspects in big data applications. This study provides a review relating to containerized environments like Docker for big data applications with load balancing. A novel scheduling mechanism of containers for big data applications established on Docker Swarm and Microservice architecture is proposed. The concept of Docker Swarm is utilized to effectively handle big data applications' workload and service discovery. Results shows that increasing workloads with respect to big data applications can be effectively managed by utilizing microservices in containerized environments and load balancing is efficiently achieved using Docker Swarm. The implementation is done using a case study deployed on a single server and then scaled to four instances. Applications developed using containerized microservices reduces average deployment time and continuous integration.https://doi.org/10.1186/s13677-022-00358-7Big dataContainerizationDockerMicroserviceDocker SwarmLoad-balancing |
| spellingShingle | Neelam Singh Yasir Hamid Sapna Juneja Gautam Srivastava Gaurav Dhiman Thippa Reddy Gadekallu Mohd Asif Shah Load balancing and service discovery using Docker Swarm for microservice based big data applications Journal of Cloud Computing: Advances, Systems and Applications Big data Containerization Docker Microservice Docker Swarm Load-balancing |
| title | Load balancing and service discovery using Docker Swarm for microservice based big data applications |
| title_full | Load balancing and service discovery using Docker Swarm for microservice based big data applications |
| title_fullStr | Load balancing and service discovery using Docker Swarm for microservice based big data applications |
| title_full_unstemmed | Load balancing and service discovery using Docker Swarm for microservice based big data applications |
| title_short | Load balancing and service discovery using Docker Swarm for microservice based big data applications |
| title_sort | load balancing and service discovery using docker swarm for microservice based big data applications |
| topic | Big data Containerization Docker Microservice Docker Swarm Load-balancing |
| url | https://doi.org/10.1186/s13677-022-00358-7 |
| work_keys_str_mv | AT neelamsingh loadbalancingandservicediscoveryusingdockerswarmformicroservicebasedbigdataapplications AT yasirhamid loadbalancingandservicediscoveryusingdockerswarmformicroservicebasedbigdataapplications AT sapnajuneja loadbalancingandservicediscoveryusingdockerswarmformicroservicebasedbigdataapplications AT gautamsrivastava loadbalancingandservicediscoveryusingdockerswarmformicroservicebasedbigdataapplications AT gauravdhiman loadbalancingandservicediscoveryusingdockerswarmformicroservicebasedbigdataapplications AT thippareddygadekallu loadbalancingandservicediscoveryusingdockerswarmformicroservicebasedbigdataapplications AT mohdasifshah loadbalancingandservicediscoveryusingdockerswarmformicroservicebasedbigdataapplications |