实验技术与管理2026，Vol.43Issue(2)：194-202,9.DOI:10.16791/j.cnki.sjg.2026.02.023
基于强化学习的机器人精准视觉操作实验方案设计
Design of an experimental scheme for robotic precise visual manipulation based on reinforcement learning
摘要
Abstract
[Objective]Reinforcement learning(RL)has emerged as a core methodology in intelligent robotics,enabling autonomous agents to interact effectively with dynamic and uncertain environments.Despite its growing importance,RL instruction often remains disconnected from practical engineering applications,which limits students'ability to translate theoretical knowledge into functional robotic manipulation systems.To address this challenge,this paper presents the design of an experimental scheme for robotic precise visual manipulation based on reinforcement learning and machine vision.The proposed scheme serves both as an instructional platform for deepening theoretical understanding and as a practical environment in which learners can engage with the complete workflow of robotic perception,decision-making,and execution in complex manipulation tasks.[Methods]A general-purpose robotic simulation platform was developed using CoppeliaSim as the core environment and integrated with a UR5 robotic manipulator and an RG2 gripper as the execution unit.An RGB-D camera was employed to acquire real-time workspace information,providing synchronized color images and per-pixel depth data,thereby substantially improving robustness in target detection and six-degree-of-freedom pose estimation.Within the perception module,RGB-D inputs were processed through height map generation,image preprocessing,and DenseNet-based feature extraction.The decision module employed a Deep Q-Network(DQN)to evaluate and select pushing and grasping actions.To explicitly connect theoretical instruction with hands-on experimentation,a dual-chain teaching framework was introduced.The theory chain focuses on control fundamentals,perception modeling,and decision optimization,while the practice chain emphasizes scene analysis,simulation-based data collection,and systematic parameter tuning.In addition,a Task Abstraction Layer(TAL)was implemented to facilitate rapid transfer of the learning framework to new manipulation tasks,such as flexible cable assembly,thereby demonstrating the platform's scalability and generalization capability.[Results]A series of simulation experiments were conducted to assess the effectiveness of the proposed experimental scheme.The results indicate that the robot successfully performed coordinated push-and-grasp operations in unstructured environments,even in the presence of significant object occlusion and scene clutter.Heatmap visualizations of learned Q-values reveal that the DQN-based decision module progressively refined its action selection strategy,converging toward optimal grasping behaviors.Parameter sensitivity analyses further show that both the discount factor and reward weighting exert a strong influence on training performance.A discount factor of 0.7 achieved the best balance between short-term and long-term rewards,yielding stable convergence and a grasping success rate exceeding 90%after 2000 training steps.Modifications to the reward function underscore the need to balance pushing and grasping incentives:eliminating pushing rewards slowed early-stage learning,whereas excessive weighting biased the policy toward pushing actions.Moreover,experiments using the TAL demonstrated that the framework could be adapted to a cable insertion task within approximately three hours,requiring fewer than 50 lines of code changes,thereby confirming its high reusability and adaptability.[Conclusions]The proposed experimental scheme effectively integrates reinforcement learning and machine vision within a hands-on educational framework for robotic manipulation.By emphasizing the explicit relationship between parameter configurations and task performance,the platform enables students to develop intuitive and systematic insights into RL algorithms and decision-making mechanisms.In addition to its instructional value,the scheme demonstrates strong technical feasibility for addressing real-world robotic manipulation challenges,providing a scalable foundation for broader applications.This work offers an innovative approach to bridging theoretical RL education and engineering practice,supporting the development of interdisciplinary talent and the construction of advanced virtual simulation laboratories.Future work will extend the platform to additional manipulation scenarios and control strategies,further strengthening its role in robotics education and research.关键词
强化学习/机器人操作/机器视觉/非结构化环境/复合型人才Key words
reinforcement learning/robot operation/machine vision/unstructured environment/interdisciplinary talents分类
信息技术与安全科学引用本文复制引用
吴细宝,丘异才,吴双双,李紫阳,陈雯柏..基于强化学习的机器人精准视觉操作实验方案设计[J].实验技术与管理,2026,43(2):194-202,9.基金项目
北京市高等教育学会2021年度立项课题"面向成果导向的创新创业教育评价机制探索与研究"(YB202190) （YB202190）
北京市高等教育本科教学改革项目"人工智能领域相关专业创新创业社会实践系列课程建设"(2022-219) （2022-219）
北京信息科技大学"勤信学者"培育计划项目(QXTCPA202102) （QXTCPA202102）
2022年度北京信息科技大学高教研究课题(2022GJYB15) （2022GJYB15）
图谱赋能的机器人课程多元化考核研究与应用(2026JGYB12) （2026JGYB12）
