计算机应用与软件2016，Vol.33Issue(5)：31-34,4.DOI:10.3969/j.issn.1000-386x.2016.05.009
Hadoop 异构集群中数据负载均衡的研究
RESEARCH ON DATA LOAD BALANCING IN HETEROGENEOUS HADOOP CLUSTER
摘要
Abstract
In Hadoop,the data load balancing has profound effect on the exertion of platform performance.First we analysed the limitation of default data load balancing,aiming at the problem of current default HDFS (Hadoop distributed file system)that the data load balancing algorithm only focuses on the storage space utilisation but not considers the heterogeneity between nodes,we presented a mathematic model which quantifies the data load balancing of heterogeneous clusters.The model calculates the theoretical space utilisation of each node based on their allocated storage space and processing capacity,and dynamically adjusts the maximum load of each node according to current average utilisation of cluster storage space.Experimental result showed that the proposed data balancing strategy could enable the heterogeneous clusters to reach more reasonable balancing state so as to improve clusters efficiency,and to decrease the execution time of job effectively as well.关键词
Hadoop/HDFS/数据负载均衡/异构集群Key words
Hadoop/HDFS/Data load balancing/Heterogeneous cluster分类
信息技术与安全科学引用本文复制引用
张松,杜庆伟,孙静,孙振..Hadoop 异构集群中数据负载均衡的研究[J].计算机应用与软件,2016,33(5):31-34,4.基金项目
国家自然科学基金项目（61202350）。 （）
