计算机应用研究2025，Vol.42Issue(6)：1624-1631,8.DOI:10.19734/j.issn.1001-3695.2024.11.0476
基于改进型多模态信息融合深度强化学习的自主超声扫描方法
Autonomous ultrasound scanning method based on improved multimodal information fusion and deep reinforcement learning
摘要
Abstract
To address the issues of low training accuracy,prolonged training time,and low success rate of scanning tasks in ultrasound scanning based on deep reinforcement learning(DRL),this paper proposed an autonomous ultrasound scanning method based on improved multimodal information fusion and DRL.Firstly,the method integrated ultrasound images,dual-view probe manipulation images,and 6D tactile feedback to provide comprehensive multimodal perception.To accurately cap-ture spatiotemporal information in multimodal data and achieve efficient feature fusion,this paper designed a multimodal fea-ture extraction and fusion module based on the self-attention mechanism(SA).Secondly,it formulated the 6D pose decision-making task for the robot as a DRL problem.And this paper designed a hybrid reward function to emulate to professional ultra-sonographers.Lastly,to address local optima and slow convergence in DRL training,this paper introduced the DSAC-PERDP algorithm.Tests in real environments demonstrate that the proposed method improves scanning accuracy,task success rate,and training speed by 49.8％,13.4％and 260.0％,respectively,compared to baseline models.Moreover,the method maintains robust performance under interference conditions.These findings validate that the proposed approach not only signi-ficantly improves scanning accuracy,task success rate,and training efficiency but also exhibits notable anti-interference cap-abilities.关键词
自主超声扫描/深度强化学习/多模态/自注意力机制/DSAC-PERDP算法Key words
autonomous ultrasound scanning/deep reinforcement learning(DRL)/multimodal/self-attention mechanism/DSAC-PERDP algorithm分类
计算机与自动化引用本文复制引用
徐加开,陆奇,李祥云,李康..基于改进型多模态信息融合深度强化学习的自主超声扫描方法[J].计算机应用研究,2025,42(6):1624-1631,8.基金项目
国家自然科学基金资助项目(51805449,62103291) （51805449,62103291）
四川省科技计划资助项目(2024YFFK0033,2023YFH0037,2023ZHCG0075,2023YFG0057,2022YFS0021,2022YFH0073) （2024YFFK0033,2023YFH0037,2023ZHCG0075,2023YFG0057,2022YFS0021,2022YFH0073）
四川大学华西医院医工交叉融合人才培养基金资助项目 （）
四川大学华西医院1·3·5卓越学科项目(ZYYC21004,ZYJC21081) （ZYYC21004,ZYJC21081）
