多模态数据融合的加工作业动态手势识别方法OA北大核心CSTPCD

In order to solve the problem of difficulty in improving the recognition accuray and the low robustness of the model caused by the lack of feature information provided by single mode data,a dynamic gesture recognition strategy based on multi-modal data fusion of machining operations for human-computer interaction was proposed.Firstly,the C3D network model was used to extract features from the depth image and color image modal data based on the spatial and temporal dimensions of videos.Secondly,the recognition results of the two modal data were fused according to the maximum principle at the decision-making level.Meanwhile,the Relu activation function used in the original model was replaced by Mish activation function to optimize the gradient update effect.Finally,through three sets of comparative experiments,it was found that the average recognition accuracy of six dynamic gestures reached 96.8%.The results showed that the proposed method achieved the goal of high accuracy and high robust-ness of dynamic gesture recognition in machining operation,which would play a role in promoting the application of human-computer interaction technology in actual production scenes.