计算机应用研究2023，Vol.40Issue(12)：3701-3705,5.DOI:10.19734/j.issn.1001-3695.2023.04.0180
基于禁忌搜索的流式计算平台负载均衡策略
Load balancing strategy of streaming computing platform based on tabu search
摘要
Abstract
Focused on the problem of unbalanced computing load distribution and low resource utilization in the native schedu-ling mechanism of big data streaming computing platform,this paper proposed a load balancing strategy based on tabu search algorithm in heterogeneous environments and applied to the Apache Flink platform.Firstly,this strategy set up a job topology model and abstracted the topology of streaming computing jobs as a directed acyclic graph.Therefore,each task slot became a node,which established the foundation for performance evaluation of computing nodes.Secondly,the method imported the performance evaluation model to nodes with performance weights in the directed acyclic graph,and obtained the performance of the nodes through normalization processing.Then the evaluation parameters were passed into the tabu search for job path optimization,so as to obtain the optimal job path.Finally,by using the CustomPatitionerWrapper interface,this strategy allocated data to the nodes included in the optimal job path and completed the balancing of computational load.The algorithm then passed evaluation parameters into the ta-bu scheduling algorithm for job path optimization,thereby obtaining the optimal job path.The experimental results show that the load balancing strategy optimized by the tabu scheduling algorithm reduces the average computing latency by 10～20 ms compared to the native Flink platform.The strategy significantly improves resource utilization,and increases average throughput by about 15％.This effectively proves the effectiveness and optimization effect of the load balancing strategy.关键词
流式计算/Apache Flink/负载均衡/性能评估/禁忌搜索算法Key words
stream computing/Apache Flink/load balancing/performance evaluation/tabu search分类
信息技术与安全科学引用本文复制引用
王英杰,李梓杨,于炯,陈鹏程..基于禁忌搜索的流式计算平台负载均衡策略[J].计算机应用研究,2023,40(12):3701-3705,5.基金项目
国家自然科学基金资助项目(62262064,62266043,61966035) （62262064,62266043,61966035）
新疆维吾尔自治区重点研发项目(2022295358) （2022295358）
新疆维吾尔自治区自然科学基金资助项目(2022D01C56) （2022D01C56）
新疆大学博士研究生创新项目(XJU2022BS072) （XJU2022BS072）
