基于表征学习的离线强化学习方法研究综述OA北大核心CSTPCD

Reinforcement learning(RL),learning an optimal policy through online interaction between an agent and environment,has recently become an important tool to solve perceptual decision-making issues in complex en-vironments.However,the online data collection may raise issues of security,time,or cost,greatly limiting the prac-tical applications of reinforcement learning.Meanwhile,tackling intricate high-dimensional data input problems has also become a significant challenge for reinforcement learning due to the intricate and multifaceted nature of raw data.Fortunately,offline reinforcement learning based on representation learning can learn the policy only from his-torical experience data without interacting with the environment.It utilizes representation learning techniques to map the features of the offline dataset into low-dimensional vectors,which are subsequently employed to train the offline reinforcement learning model.This data-driven paradigm provides a new opportunity to realize the general artificial intelligence.To this end,this paper comprehensively reviews the recent research on offline reinforcement learning based on representation learning.Firstly,the problem setup of offline reinforcement learning is given.Then,the existing technologies are summarized from three aspects:Methodologies,benchmarks,offline policy evaluation and hyperparameter selection.Moreover,the study trends of offline reinforcement learning in industries,recom-mendation systems,intelligent driving,and other fields are introduced.Finally,the conclusion is drawn and the key challenges and development trends of offline reinforcement learning based on representation learning in the future are discussed,so as to provide a valuable reference for subsequent study.