工程科学学报2024，Vol.46Issue(9)：1521-1534,14.DOI:10.13374/j.issn2095-9389.2023.07.30.001
无人集群系统深度强化学习控制研究进展
Deep reinforcement learning to control an unmanned swarm system
摘要
Abstract
Recently, testing and using micro-unmanned vehicles, such as unmanned aerial vehicles (UAVs), in scenarios such as supply transportation, agricultural management, and military operations have become more common. It is no longer sufficient to control a single UAV to accomplish all missions. With the increasing complexities associated with operating and task requirements, an unmanned swarm requires a series of algorithms with higher efficiency, greater generalization ability, and better adaptability than the earlier algorithms. A combination of unmanned swarms with artificial intelligence is becoming a common solution to manage the above requirements. Deep reinforcement learning (DRL) is a machine learning method that combines deep learning (DL) and reinforcement learning (RL); therefore, this method has the advantages of DL and RL. Using an RL method, an agent can learn from the environment by trial and error and make decisions that autonomously obtain high scores. However, when the given environment is complex, the decision function of the agent may be too difficult to implement and then the agent cannot make the correct decision. The DL method has strong fitting ability. A suitable deep neural network can simulate any linear or nonlinear function. If the DL method is used to simulate the decision function in RL, the hybrid method can solve the problem that an agent cannot solve and make a correct decision in a complex environment. The combination of an unmanned swarm and a DRL method has been widely studied. This paper introduces the concept of DRL from the perspective of principles and characteristics. This paper analyzes several typical DRL algorithms, discusses the various control requirements of a UAV swarm, and then focuses on the achievements of combining DRL and a UAV swarm control. Finally, this paper presents viewpoints on the application prospects and challenges related to landing and transformation in the combination field. The concept of an unmanned swarm originated from the study of the behavior of biological groups. Several species of bees, ants, birds, fish, and other creatures exhibit complex group behaviors. These clusters comprise many independent individuals in accordance with certain aggregation rules to form a coordinated, orderly group movement mechanism. Similar to biological clusters, in the field of robotics or UAVs, unmanned swarm systems are crowded intelligent systems. These systems consist of multiple homogeneous or heterogeneous unmanned equipment to achieve mutual behavior coordination and jointly complete specific tasks through interactive feedback and incentive response of information. In practical applications, an unmanned swarm system needs to meet the requirements of an open environment, a changeable situation, limited resources, and real-time responses. This system needs to have multicore collaborative capabilities such as distributed collaborative perception, intelligent collaborative decision-making, and robust collaborative control. The distributed intelligent collaborative control method based on DRL can fully meet the control requirements of high intelligence and robustness of unmanned cluster systems.关键词
无人集群/集群控制/深度强化学习/多智能体/人工智能/集群智能Key words
unmanned swarm/swarm control/deep reinforcement learning/multiagent/artificial intelligence/swarm intelligence分类
航空航天引用本文复制引用
梁鸿涛,王耀南,华和安,钟杭,郑成宏,曾俊豪,梁嘉诚,李政辰..无人集群系统深度强化学习控制研究进展[J].工程科学学报,2024,46(9):1521-1534,14.基金项目
湖南省自然科学基金重大项目(2021JC0004) （2021JC0004）
国家重点研发计划资助项目(2022YFB4701800,2021ZD0114503) （2022YFB4701800,2021ZD0114503）
湖南省自然科学基金资助项目(2023JJ40165) （2023JJ40165）
国家自然科学基金资助项目(62173132) （62173132）
