Dynamic load balancing algorithm for large data flow in distributed complex networks
Information society brings convenience to people, but also produces a lot of data. Relational databases are not suitable for processing big data due to architecture defects. The most commonly used system to store and process large amounts of data is the NoSQL (Not only Structured Query Language) dat...
| Main Author: | |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
De Gruyter
2018-11-01
|
| Series: | Open Physics |
| Subjects: | |
| Online Access: | https://doi.org/10.1515/phys-2018-0089 |
| _version_ | 1830509083914928128 |
|---|---|
| author | Zhang Zhuo |
| author_facet | Zhang Zhuo |
| author_sort | Zhang Zhuo |
| collection | DOAJ |
| description | Information society brings convenience to people, but also produces a lot of data. Relational databases are not suitable for processing big data due to architecture defects. The most commonly used system to store and process large amounts of data is the NoSQL (Not only Structured Query Language) database. Obviously, it is very important to cooperate with these independent computers to accomplish processing tasks efficiently, which is the function of load balancing. This paper studies the commonly used NoSQL database and load balancing algorithms, and designs and implements a more efficient load balancing algorithm. By introducing the relationship between nodes and the children of their brother nodes, we reduce the height of the whole sorted binary tree. The time cost of the algorithm is reduced versus the commonly used weighted polling algorithm O(N) to O(log N), while the spatial cost remains unchanged. The equalization algorithm synthetically utilizes the characteristics of big data processing systems and has good performance. At the same time, the algorithm can quickly find the sub-optimal nodes when the optimal nodes have been occupied, so it is very suitable for load balancing in highly concurrent systems. Finally, the effectiveness of the proposed load balancing algorithm is verified by simulation. |
| first_indexed | 2024-12-22T01:29:18Z |
| format | Article |
| id | doaj.art-8db2c7f3f0d14cee9a2dec12eb03731f |
| institution | Directory Open Access Journal |
| issn | 2391-5471 |
| language | English |
| last_indexed | 2024-12-22T01:29:18Z |
| publishDate | 2018-11-01 |
| publisher | De Gruyter |
| record_format | Article |
| series | Open Physics |
| spelling | doaj.art-8db2c7f3f0d14cee9a2dec12eb03731f2022-12-21T18:43:32ZengDe GruyterOpen Physics2391-54712018-11-0116170671610.1515/phys-2018-0089phys-2018-0089Dynamic load balancing algorithm for large data flow in distributed complex networksZhang Zhuo0Department of Statistics, Heihe University, Heihe, ChinaInformation society brings convenience to people, but also produces a lot of data. Relational databases are not suitable for processing big data due to architecture defects. The most commonly used system to store and process large amounts of data is the NoSQL (Not only Structured Query Language) database. Obviously, it is very important to cooperate with these independent computers to accomplish processing tasks efficiently, which is the function of load balancing. This paper studies the commonly used NoSQL database and load balancing algorithms, and designs and implements a more efficient load balancing algorithm. By introducing the relationship between nodes and the children of their brother nodes, we reduce the height of the whole sorted binary tree. The time cost of the algorithm is reduced versus the commonly used weighted polling algorithm O(N) to O(log N), while the spatial cost remains unchanged. The equalization algorithm synthetically utilizes the characteristics of big data processing systems and has good performance. At the same time, the algorithm can quickly find the sub-optimal nodes when the optimal nodes have been occupied, so it is very suitable for load balancing in highly concurrent systems. Finally, the effectiveness of the proposed load balancing algorithm is verified by simulation.https://doi.org/10.1515/phys-2018-0089nosqldatabasesload balancing algorithm, distributed mass data storage systemsorted binary tree05.90.+m07.05.tp |
| spellingShingle | Zhang Zhuo Dynamic load balancing algorithm for large data flow in distributed complex networks Open Physics nosql databases load balancing algorithm, distributed mass data storage system sorted binary tree 05.90.+m 07.05.tp |
| title | Dynamic load balancing algorithm for large data flow in distributed complex networks |
| title_full | Dynamic load balancing algorithm for large data flow in distributed complex networks |
| title_fullStr | Dynamic load balancing algorithm for large data flow in distributed complex networks |
| title_full_unstemmed | Dynamic load balancing algorithm for large data flow in distributed complex networks |
| title_short | Dynamic load balancing algorithm for large data flow in distributed complex networks |
| title_sort | dynamic load balancing algorithm for large data flow in distributed complex networks |
| topic | nosql databases load balancing algorithm, distributed mass data storage system sorted binary tree 05.90.+m 07.05.tp |
| url | https://doi.org/10.1515/phys-2018-0089 |
| work_keys_str_mv | AT zhangzhuo dynamicloadbalancingalgorithmforlargedataflowindistributedcomplexnetworks |