计算机应用研究2017，Vol.34Issue(3)：727-729,755,4.DOI:10.3969/j.issn.1001-3695.2017.03.020
基于logistic回归模型的Hadoop本地任务调度优化算法
Hadoop local tasks scheduling optimization algorithm based on logistic regression model
摘要
Abstract
For a taskTracker has multiple local tasks available,by default,the scheduler executes those tasks in succession with the order of the tasks to be found,this is inefficient.In order to optimize the local tasks scheduling,this paper presented Hadoop local tasks scheduling optimization algorithm based on machine learning.First,it selected and defined related feature vectors of the local tasks.Then,based on the way of machine learning with logistic regression model,it trained these vectors to get the weight of each vector to decide the task priority,and updated the model constantly by the overload rules.The experimental results show that the proposed algorithm improves map task data locality,at the same time it reduces job running time.关键词
Hadoop/MapReduce/本地调度/任务优先级/过载规则/logistic回归模型Key words
Hadoop/MapReduce/local tasks scheduling/task priority/overload rules/logistic regression model分类
信息技术与安全科学引用本文复制引用
帅仁俊,沈阳,陈平,潘静,董亚楠..基于logistic回归模型的Hadoop本地任务调度优化算法[J].计算机应用研究,2017,34(3):727-729,755,4.基金项目
国家公益性科研专项项目(201310162,201210022) （201310162,201210022）
连云港科技支撑计划资助项目(SH1110) （SH1110）
