高技术通讯2017，Vol.27Issue(4)：295-302,8.DOI:10.3772/j.issn.1002-0470.2017.04.001
面向多用户环境的MapReduce集群调度算法研究
Research on a job scheduling algorithm for multi user MapReduce clusters
摘要
Abstract
To solve the problem that the existing MapReduce scheduling strategy cannot realize dynamic resource allocation according to user''s actual resource demand in a multi-user environment, an algorithm for MapReduce clusters scheduling based on historical execution information(HEI), called the HEI scheduler, was proposed.The algorithm obtains the rules of the variation of each user group''s resource demand with time by establishing the mechanism for collection and analysis of cluster operation''s execution information, and uses operation''s actual slot occupying time to measure the occupied resource of operation to dynamically determine the minimum shared resource and weights of cluster remaining resource distribution.The experimental result indicates that the proposed execution information analysis mechanism can describe the resource demand of operation exactly.The MapReduce cluster scheduling algorithm based on historical execution information can effectively reduce the overall operation execution time.关键词
MapReduce集群/多用户环境/调度算法/作业执行信息收集Key words
MapReduce cluster/multi-user environment/scheduling algorithm/job implementation information collection引用本文复制引用
陈重韬..面向多用户环境的MapReduce集群调度算法研究[J].高技术通讯,2017,27(4):295-302,8.基金项目
国家科技支撑计划(2012BAH46B03),国家自然科学基金(61402473),核高基(2013ZX01039-002-001-001)和中国科学院先导专项(XDA06030200)资助项目. （2012BAH46B03）
