计算机应用与软件2024，Vol.41Issue(8)：46-52,7.DOI:10.3969/j.issn.1000-386x.2024.08.007
面向负载均衡的动态均衡分区策略
DYNAMIC BALANCING PARTITION STRATEGY FOR LOAD BALANCING
摘要
Abstract
In view of the unbalanced load on the reduce side caused by MapReduce computing framework processing inclined data sets,this paper proposes a dynamic balanced partition strategy.In the mapper stage,the data segmentation principle based on divide and conquer method was proposed to process the incoming data groups.The dynamic allocation principle was designed combined with the idea of the best adaptive algorithm to gradually allocate the segmented data blocks to the server.According to the partition index,it was allocated to each reduce node to achieve load balancing.The experimental results show that,compared with the two benchmark models,the average task execution time of the dynamic balanced partition strategy is reduced by 7.7％,which indicates that the dynamic balanced partition strategy can better solve the problem of data skew,reduce the task execution time,and verify the effectiveness of the model.关键词
MapReduce/负载均衡/数据倾斜/数据分区Key words
MapReduce/Load balancing/Data skew/Data partition分类
计算机与自动化引用本文复制引用
杨迪,赵家伟,王鹏,赵建平..面向负载均衡的动态均衡分区策略[J].计算机应用与软件,2024,41(8):46-52,7.基金项目
中央引导地方科技发展资金吉林省基础研究专项(202002038JC). （202002038JC）
