工程科学学报2024，Vol.46Issue(8)：1426-1433,8.DOI:10.13374/j.issn2095-9389.2023.07.01.002
基于多模态信息融合的四足机器人避障方法
Obstacle avoidance approach for quadruped robot based on multi-modal information fusion
摘要
Abstract
This paper proposes a multimodal information fusion neural network model that integrates visual,radar,and proprioceptive information.The model uses a spatial crossmodal attention mechanism to fuse the information,allowing the robot to focus on the most relevant information for obstacle avoidance.The attention mechanism enables the robot to selectively focus on the most relevant informative sensory inputs,which improves its ability to navigate complex terrain.The proposed method was evaluated using multiple experiments in challenging simulated environments,and the results showed a significant improvement in the obstacle avoidance success rate.The proposed method uses an actor-critic architecture and a proximal policy optimization(PPO)algorithm to train the robot in a simulated environment.The training process aims to reduce the difference between the robot's performance in simulated and real-world environments.To achieve this,we randomly adjust the simulation environment's parameters and add random noise to the robot's sensory inputs.This approach allows the robot to learn a robust planning strategy that can be deployed in real-world environments.The multimodal information fusion neural network model is designed using a transformer-based architecture.The model shares the encoding of three types of tokens and generates features for the robot's proprioceptive,visual,and point cloud inputs.The transformer encoder layers are stacked such that the token information from the three modalities can be fuzed at multiple levels.To balance the information from the three modalities,we first separately collect information for each modality and calculate the average value of all tokens from the same modality to obtain a single feature vector.This multimodal information fusion approach improves the robot's decision-making capabilities in complex environments.The novelty of the proposed method lies in the introduction of a spatial crossmodal attention mechanism that allows the robot to selectively attend to the most informative sensory inputs.This attention mechanism improves the robot's ability to navigate complex terrain and provides a certain degree of reliability for the quadruped robot in dynamic unknown environments.The combination of multimodal information fusion and attention mechanism enables the robot to adapt better to complex environments,thus improving its obstacle avoidance capabilities.Therefore,the proposed method provides a promising approach for improving the obstacle avoidance capabilities of quadruped robots in complex environments.The proposed method is based on the multimodal information fusion neural network model and spatial crossmodal attention mechanism.The experimental results demonstrate the effectiveness of the proposed method in improving the robot's obstacle avoidance success rate.Moreover,the potential applications of the proposed method include search and rescue missions,exploration,and surveillance in complex environments.关键词
四足机器人/路径规划/强化学习/信息融合/TransformerKey words
quadruped robot/path planning/reinforcement learning/information fusion/transformer分类
信息技术与安全科学引用本文复制引用
吕友豪,贾袁骏,庄圆,董琦..基于多模态信息融合的四足机器人避障方法[J].工程科学学报,2024,46(8):1426-1433,8.基金项目
网络空间安全态势感知与评估安徽省重点实验室开放课题(CSSAE-2021-003) （CSSAE-2021-003）
国家自然科学基金项目青年科学基金项目(61803353) （61803353）
