重庆大学学报2025，Vol.48Issue(6)：63-73,11.DOI:10.11835/j.issn.1000-582X.2024.008
基于双流互信息联合匹配的小样本行为识别
Two-stream joint matching based on mutual information for few-shot action recognition
摘要
Abstract
Although few-shot action recognition based on the metric learning paradigm has achieved significant success,it fails to address the following issues:1)inadequate action relation modeling and underutilization of multi-modal information;2)challenges in handling video matching problems with different lengths and speeds,and misaligned video sub-actions.To address these limitations,we propose a two-stream joint matching(TSJM)method based on mutual information,which consists of two modules:multi-modal contrastive learning module(MCL)and joint matching module(JMM).The MCL extensively explores inter-modal mutual information relationships,and thoroughly extracts modal information to enhance the modeling of action relationships.The JMM is primarily designed to simultaneously solve the aforementioned video matching problems.By integrating dynamic time warping(DTW)and bipartite graph matching,it optimizes the matching process to generate the final alignment results,thereby achieving high few-shot action recognition accuracy.We evaluate the proposed method on two widely used few-shot action recognition datasets(SSV2 and Kinetics),and conduct comprehensive ablation experiments to substantiate the efficacy of our approach.关键词
深度学习/行为识别/多模态/小样本学习Key words
deep learning/action recognition/multimodal/few-shot learning分类
计算机与自动化引用本文复制引用
邓龙,冯波,葛永新..基于双流互信息联合匹配的小样本行为识别[J].重庆大学学报,2025,48(6):63-73,11.基金项目
重庆市技术创新与应用发展专项(CSTB2022TIAD-KPX0100) （CSTB2022TIAD-KPX0100）
国家自然科学基金(62176031) （62176031）
中央高校基本科研业务费专项(2023CDJYGRHZD05).Supported by the Specialized Project for Technology Innovation and Application Development of Chongqing(CSTB2022TIAD-KPX0100),National Natural Science Foundation of China(62176031),and the Fundamental Research Funds for the Central Universities(2023CDJYGRHZD05). （2023CDJYGRHZD05）
