浙江大学学报（理学版）2026，Vol.53Issue(2)：172-180,9.DOI:10.3785/j.issn.1008-9497.25115
基于关键点感知和窗口-时序注意力Transformer的模糊手势重建方法
Blurry hand reconstruction based on key point perception and window-temporal attention Transformer
连远锋 1王鑫 2崔超 3程岩斌 3夏曦乐 2赵刚强3
作者信息
- 1. 中国石油大学(北京)人工智能学院,北京 102249
- 2. 中国石油大学(北京)人工智能学院,北京 102249||北京虚拟动点科技有限公司,北京 100040
- 3. 北京虚拟动点科技有限公司,北京 100040
- 折叠
摘要
Abstract
Reconstructing hand gestures from ambiguous images caused by rapid hand motion is of great significance.In order to avoid the ambiguity caused by blurred gesture images,this paper proposes an improved approach based on key point perception and window-temporal attention.First,the method utilizes residual networks and feature pyramids to extract key point features of the hand from dynamic fuzzy gesture images;then,with the proposed window-temporal attention Transformer,the spatial information within a single frame is captured by using the shifted window-based multi-head self-attention(SW-MSA),and a novel inter-frame temporal attention(FTA)mechanism is introduced to explicitly model the correlation of multi-frames across the time-steps to effectively fuse the spatial and temporal information so as to solve the ambiguity of the hand motion.In addition,considering that different hand joints have different motion sensitivities,a weighted strategy based on motion analysis of key points is introduced in the training to better constrain the reconstruction process.Results on the BlurHand dataset show that the proposed method can significantly improve the accuracy of re-constructing 3D hand sequences from a single blurred image compared to existing methods.关键词
模糊手势重建/关键点感知/窗口-时序注意力/序列重建Key words
blurry hand reconstruction/key point perception/window-temporal attention/sequence reconstruction分类
信息技术与安全科学引用本文复制引用
连远锋,王鑫,崔超,程岩斌,夏曦乐,赵刚强..基于关键点感知和窗口-时序注意力Transformer的模糊手势重建方法[J].浙江大学学报（理学版）,2026,53(2):172-180,9.
