计算机工程与应用2026,Vol.62Issue(18):98-111,14.DOI:10.3778/j.issn.1002-8331.2601-0088
语义-几何协同引导的解耦式语言驱动六自由度抓取方法
Semantic-Geometric Synergistically Guided Decoupled Language-Driven 6-DoF Grasping Method
摘要
Abstract
To address the semantic and geometric feature entanglement caused by implicit coupling mechanisms,as well as the spatial misalignment between 2D visual perception and 3D geometric reasoning in language-driven 6-Degree-of-Freedom(6-DoF)grasping tasks within unstructured open scenarios,this paper proposes a decoupled language-driven grasping method guided by semantic-geometric synergy.The proposed method explicitly decouples the task into two syn-ergistic stages:cross-modal visual grounding and local geometric reasoning.In the visual grounding stage,a CLIP visual adapter is designed to introduce alignment priors,thereby achieving front-end cross-modal feature alignment.Simultane-ously,a bidirectional encoder is employed for iterative feature refinement,effectively enhancing the model's capability to parse complex instructions.In the geometric reasoning stage,a semantics-guided geometric cascade strategy is adopted.By leveraging high-confidence masks from the grounding stage for semantic-level cropping in the point cloud space,the pre-trained FGC-GraspNet is guided to generate grasp poses within the reconstructed local target point cloud space,effec-tively mitigating environmental noise interference.For the visual grounding task,the proposed model achieves intersec-tion over union(IoU)scores of 94.91%and 90.03%on the OCID-VLG and the seen-category test set of RoboReflt,respectively,outperforming HiFi-CS by 6.65 percentage points on OCID-VLG.Real-world robotic arm grasping experi-ments further validate the effectiveness of the decoupled framework in optimizing geometric inputs:in simple,complex,and challenging scenes,the system's grasping success rates for seen objects reach 97.5%,90.0%,and 90.0%,respectively,while the success rates for unseen objects reach 86.7%,73.3%,and 67.6%,respectively.Meanwhile,compared with existing end-to-end methods,the decoupled framework demonstrates higher robustness in multi-interference environments.Addi-tionally,the system achieves an inference speed of approximately 5.34 FPS with low computational overhead,validating its feasibility for deployment on edge devices.关键词
6-DoF抓取/视觉定位/跨模态对齐/双向交叉注意力Key words
6-DoF grasping/visual grounding/cross-modal alignment/bidirectional cross-attention分类
信息技术与安全科学引用本文复制引用
李西兴,朱道美,张杨,刘涛,胡佑希..语义-几何协同引导的解耦式语言驱动六自由度抓取方法[J].计算机工程与应用,2026,62(18):98-111,14.基金项目
国家自然科学基金(62403186) (62403186)
国家留学基金委资助(202508420143) (202508420143)
湖北省重点研发计划项目(2025BAB063) (2025BAB063)
湖北省自然科学基金(2024AFB153) (2024AFB153)
湖北省科技支撑计划项目(2024BAB058). (2024BAB058)