计算机应用与软件2018，Vol.35Issue(5)：149-156,201,9.DOI:10.3969/j.issn.1000-386x.2018.05.027
基于预测模型的HDFS集群负载均衡优化与研究
OPTIMIZATION AND RESEARCH OF HDFS CLUSTER LOAD BALANCING BASED ON PREDICTION MODEL
于磊春 1陈健美 1刘响 1胡杨1
作者信息
- 1. 江苏大学计算机科学与通信工程学院 江苏镇江212013
- 折叠
摘要
Abstract
In view of the subjectivity, lagness and singleness of metric of the decision threshold in the default load balancing strategy of HDFS cluster, by analyzing the characteristics of HDFS clustering architecture, HDFS cluster processing objects and the cluster itself dynamic real-time changes, combined with some of the load balancing strategy threshold improvement algorithm,this paper presented a prediction model based on the estimated file attributes combined with cluster attributes comprehensive calculation threshold method.The threshold calculated by this method was substituted into the corresponding load balancing strategy for load optimization.The experimental results showed that the proposed method based on the estimated model to estimate the file attributes had high accuracy and the load balancing strategy based on the prediction model was efficient in adjusting the load of the cluster.At the same time,it could further shorten the execution time of cluster operations and improved the efficiency of cluster operations.关键词
HDFS集群/预测模型/文件属性/集群属性/阈值/负载均衡Key words
HDFS cluster/Prediction model/File attributes/Cluster attributes/Threshold Load balancing分类
信息技术与安全科学引用本文复制引用
于磊春,陈健美,刘响,胡杨..基于预测模型的HDFS集群负载均衡优化与研究[J].计算机应用与软件,2018,35(5):149-156,201,9.
