液晶与显示2025，Vol.40Issue(6)：867-880,14.DOI:10.37188/CJLCD.2024-0345
基于自适应遮挡恢复与拓扑姿态双向感知的三维手部重建方法
3D hand reconstruction method based on adaptive occlusion recovery and topology-pose bidirectional perception
摘要
Abstract
Existing 3D hand mesh reconstruction methods face multiple challenges,especially when dealing with occlusion and highly flexible hand poses,which lead to issues such as missing geometric information and topological errors.To enable accurate and efficient 3D hand reconstruction under occlusion,this paper proposes a two-stage network framework for real-time and efficient reconstruction of hand 3D meshes from monocular RGB images.In the first stage,the adaptive occlusion recovery module is designed by introducing learnable attention weight masks and region consistency loss within the attention mechanism.This module targets occluded regions and adaptively recovers information,significantly enhancing feature representation ability under occlusion.In the second stage,the paper combines static topology modeling and dynamic pose perception,as well as feature information exchange between bidirectional graph convolutions and a novel joint rotation-aware attention.This results in the topology-pose bidirectional perception module,which achieves complementary enhancement of static and dynamic features,improving the ability to capture fine-grained joint details.The proposed method is evaluated through qualitative and quantitative experiments on the FreiHAND and InterHand2.6M datasets,compared with state-of-the-art methods.Experimental results show that on the FreiHAND dataset,the proposed method achieves the PA-MPVPE reduction to 6.1 mm with an inference speed of 39 FPS,and on the InterHand2.6M dataset,the MPJPE of the proposed method is reduced to 8.07 mm,and the MPVPE is reduced to 8.22 mm.The proposed approach meets the requirements for robust occlusion handling,real-time performance,and accurate pose estimation in 3D hand reconstruction.关键词
三维手部网格重建/注意力机制/图卷积网络/区域一致性损失/深度学习Key words
3D hand mesh reconstruction/attention mechanism/graph convolutional network/regional consistency loss/deep learning分类
计算机与自动化引用本文复制引用
刘佳,黄楠瑄,陈大鹏,魏李娜..基于自适应遮挡恢复与拓扑姿态双向感知的三维手部重建方法[J].液晶与显示,2025,40(6):867-880,14.基金项目
国家自然科学基金(No.62473200,No.62476238) （No.62473200,No.62476238）
江苏省青年科技人才托举工程项目(No.JSTJ-2024-195)Supported by National Natural Science Foundation of China(No.62473200,No.62476238) （No.JSTJ-2024-195）
Jiangsu Province Youth Science and Technology Talent Support Project(No.JSTJ-2024-195) （No.JSTJ-2024-195）
