中山大学学报（自然科学版）Issue(6)：55-66,12.DOI:10.13471/j.cnki.acta.snus.2015.06.011
适应节能与异构环境的 MapReduce 数据布局策略
An Energy-Efficient and Heterogeneous Environment Adaptive Data Layout Strategy for MapReduce
摘要
Abstract
The problem of high energy consumption producing from big data processing is an important issue that needs to be solved, especially under the background of data explosion.Based on analyzing problems of the existing data layout policy, the problems of the in adaptation of energy-saving mode based on storage area division and heterogeneous HDFS cluster, the inflexibility of data block segmentation al-gorithm, the randomness of storage node selection, proposing a data layout strategy orienting to energy conservation are analyzed.Firstly, the new strategy divides the cluster into two different storage areas to meet the needs of saving energy:Active-Zone and Sleep-Zone;secondly, the new strategy has made im-provements on traditional data block computing method, proposes a minimum number of jobs calculation method to determine the number of data blocks;at last, the new strategy can increase the adaptability of the heterogeneous cluster environment and can choose the appropriate storage nodes according to different job types.Experimental results show that the new data layout strategy can adapt to the heterogeneous cluster environment and reach the goal of reducing energy consumption for MapReduce jobs.关键词
绿色计算/MapReduce/异构环境/数据布局Key words
green computing/MapReduce/heterogeneous environment/data layout分类
计算机与自动化引用本文复制引用
廖彬,张陶,于炯,刘继,钟磊,刘炎..适应节能与异构环境的 MapReduce 数据布局策略[J].中山大学学报（自然科学版）,2015,(6):55-66,12.基金项目
国家自然科学基金资助项目（61562078，61262088，71261025）；新疆财经大学博士启动基金资助项目 （）
