运筹与管理2024，Vol.33Issue(9)：160-167,8.DOI:10.12005/orms.2024.0300
基于深度强化学习技术的算力服务平台革新
Innovation in Deep Reinforcement Learning Based Computing Capacity Service Platform:Case Analysis of the Project to Channel Computing Resources from the East to the West
摘要
Abstract
To meet the huge demand of information industry for computing capacity,China has initiated a project to channel computing resources from the east to the west,which enables large-scare computing capacity schedu-ling.The computing capacity service platform is the key part of the project which is responsible for providing computing capacity to support massive concurrent services.It is faced with the challenge of heavy processing burden but limited computing capacity resources for delay-sensitive services.In this paper,we aim to study the innovation in deep reinforcement learning based computing capacity service platform.The significance of the research can be summarized as the following three points.Firstly,this paper is the first academic paper to study the core technology of computing capacity service platform.Starting from the challenges and practical background faced by the computing capacity service platform,the paper explores its inherent complex system scheduling and management problems.Secondly,a deep reinforcement learning based computing capacity resource providing strategy is designed for computing capacity service platforms.Thirdly,the performance of the provided strategies is verified through simulations.Countermeasures and suggestions for the subsequent design and construction of the computing capacity service platform are proposed based on simulation results. The deep reinforcement learning based computing capacity providing strategy is also designed for service platform management.Firstly,a Hidden Markov Model based model for computing capacity chain is proposed considering energy consumption,delay and bandwidth as a multi-objective.The model is proposed mainly for de-lay sensitive services and considers the processing capacity limitations.Multiple sub-tasks of a computing task are deployed on the computing capacity chain composed of multiple edge network nodes according to network environment changes based on a certain scheduling strategy.Meanwhile,indicators such as energy consumption,delay and bandwidth are considered to optimize the selection of computing capacity providing nodes and the path of computing capacity chain collaboratively.Secondly,the improved list Viterbi based prioritized replay double deep Q network algorithm(VPDDQN)is designed which considers the complex and changing network environ-ment and the huge scheduling action space.VPDDQN is mainly composed of two steps.(1)The improved list Viterbi algorithm selects several optimal candidate scheduling solutions corresponding to a certain state in order to accelerate model training speed and reduce model training difficulty.(2)DDQN,which is a kind of deep reinforcement learning algorithms,is improved to select the best scheduling solution of the candidate solutions.In this way,the proposed strategy can make the optimal scheduling and optimization solution for edge computing capacity chain according the state of network environment. Taking the project to channel computing resources from the east to the west as a case background,we select two cities in Beijing-Tianjin-Hebei region as the simulation scenario.The simulation results are as follows.Firstly,the proposed algorithm VPDDQN,which is used as computing capacity providing strategy for the project,is efficient and the model training time is short for VPDDQN.Secondly,the proposed algorithm has the best overall performance of the other benchmark algorithms in terms of delay,bandwidth and energy consumption.Thirdly,the proposed computing capacity providing strategy can achieve the maximum completion rate of compu-ting tasks for different scales of computing tasks,thereby improving the economic efficiency of computing capacity network operation.The simulation results show that the proposed strategy can help the platform improve the performance and economy of computing capacity provision. Finally,we provide policy implications and suggestions for building the computing capacity service platform from the perspectives of intercity computing capacity collaboration,information collection and regional tasks planning.Firstly,reasonable and efficient computing capacity providing strategies have a crucial positive impact on the construction and operation of the project.Secondly,the management department needs to recognize the complexity of the multi-point collaborative computing capacity chain providing method and collect real-time infor-mation about the delay,bandwidth and energy consumption of geographically distributed edge computing sites.Thirdly,task plans should be made according to local conditions to improve completion rate of computing tasks and the economy of computing power network operation.关键词
算力服务平台/东数西算/深度强化学习Key words
computing capacity service platform/channel computing resources from the east to the west/deep reinforcement learning分类
管理科学引用本文复制引用
李泰新,刘锋,徐健..基于深度强化学习技术的算力服务平台革新[J].运筹与管理,2024,33(9):160-167,8.基金项目
辽宁省社会科学规划基金项目(L22BGL021) （L22BGL021）
