通信学报2024，Vol.45Issue(12)：28-43,16.DOI:10.11959/j.issn.1000-436x.2024268
基于增强负例多粒度区分模型的视频动作识别研究
Study on video action recognition based on augment negative example multi-granularity discrimination model
摘要
Abstract
An augment negative example discrimination paradigm based on contrastive learning was proposed to im-prove the model's fine-grained discrimination ability of video actions.The most challenging video-text negative pairs was generated,forming an augmented negative example set for each video sample.Based on this paradigm,a multi-granularity discrimination model for video action recognition was proposed to further distinguish between positive and negative examples.In this model,video features were extracted by the video representation module guided by textual positive examples,while self-correlation relationships between positive and negative semantics were established by the semantic discriminator equipped with a self-attention mechanism.Meanwhile,a coarse-grained distinction between the video modality and the augmented negative example set was achieved,while a fine-grained distinction between positive examples and the augmented negative example set within the text modality was also accomplished.Experimental results demonstrate that the augment negative set improves the model's recognition ability on fine-grained class labels,and the multi-granularity discrimination model outperforms current state-of-the-art methods on the Kinetics-400,HMDB51 and UCF101 datasets.关键词
对比学习/增强负例/范式/视频动作识别Key words
contrastive learning/augmented negative examples/paradigm/video action recognition分类
信息技术与安全科学引用本文复制引用
刘良振,杨阳,夏莹杰,邝砾..基于增强负例多粒度区分模型的视频动作识别研究[J].通信学报,2024,45(12):28-43,16.基金项目
国家重点研发计划基金资助项目(No.2022YFF0902500) The National Key Research and Development Program of China(No.2022YFF0902500) （No.2022YFF0902500）
