郑州大学学报（工学版）2024，Vol.45Issue(5)：30-36,7.DOI:10.13705/j.issn.1671-6833.2024.02.007
多模态数据融合的加工作业动态手势识别方法
Dynamic Gesture Recognition Method for Machining Operations Based on Multi-modal Data Fusion
摘要
Abstract
In order to solve the problem of difficulty in improving the recognition accuray and the low robustness of the model caused by the lack of feature information provided by single mode data,a dynamic gesture recognition strategy based on multi-modal data fusion of machining operations for human-computer interaction was proposed.Firstly,the C3D network model was used to extract features from the depth image and color image modal data based on the spatial and temporal dimensions of videos.Secondly,the recognition results of the two modal data were fused according to the maximum principle at the decision-making level.Meanwhile,the Relu activation function used in the original model was replaced by Mish activation function to optimize the gradient update effect.Finally,through three sets of comparative experiments,it was found that the average recognition accuracy of six dynamic gestures reached 96.8%.The results showed that the proposed method achieved the goal of high accuracy and high robust-ness of dynamic gesture recognition in machining operation,which would play a role in promoting the application of human-computer interaction technology in actual production scenes.关键词
多模态数据融合/加工作业/动态手势识别/C3D/Mish激活函数/人机交互Key words
multi-modal data fusion/machining operation/dynamic gesture recognition/C3D/Mish activation function/human-computer interaction分类
机械制造引用本文复制引用
张富强,曾夏,白筠妍,丁凯..多模态数据融合的加工作业动态手势识别方法[J].郑州大学学报（工学版）,2024,45(5):30-36,7.基金项目
国家重点研发计划项目(2021YFB3301702) （2021YFB3301702）
陕西省科技重大专项(2018zdzx01-01-01) （2018zdzx01-01-01）
