计算机工程与应用2024，Vol.60Issue(9)：1-18,18.DOI:10.3778/j.issn.1002-8331.2310-0090
基于多模态数据的人体行为识别方法研究综述
Review on Human Action Recognition Methods Based on Multimodal Data
摘要
Abstract
Human action recognition(HAR)is widely applied in the fields of intelligent security,autonomous driving and human-computer interaction.With advances in capture equipment and sensor technology,the data that can be acquired for HAR is no longer limited to RGB data,but also multimodal data such as depth,skeleton,and infrared data.Feature extrac-tion methods in HAR based on RGB and skeleton data modalities are introduced in detail,including handcrafted-based and deep learning-based methods.For RGB data modalities,feature extraction algorithms based on two-stream convolu-tional neural network(2s-CNN),3D convolutional neural network(3DCNN)and hybrid network are analyzed.For skele-ton data modalities,some popular pose estimation algorithms for single and multi-person are firstly introduced.The classi-fication algorithms based on convolutional neural network(CNN),recurrent neural network(RNN),and graph convolu-tional neural network(GCN)are analyzed stressfully.A further comprehensive demonstration of the common datasets for both data modalities is presented.In addition,the current challenges are explored based on the corresponding data struc-ture features of RGB and skeleton.Finally,future research directions for deep learning-based HAR methods are discussed.关键词
视频理解/人体行为识别/深度学习/特征提取/姿态评估算法Key words
video understanding/human action recognition/deep learning/feature extraction/pose estimation algorithms分类
信息技术与安全科学引用本文复制引用
王彩玲,闫晶晶,张智栋..基于多模态数据的人体行为识别方法研究综述[J].计算机工程与应用,2024,60(9):1-18,18.基金项目
国家自然科学基金(62276213). （62276213）
