基于深度强化学习的混杂场景目标物体推抓协同策略

doi:10.3969/j.issn.1004-132X.2025.01.014

中国机械工程 ›› 2025, Vol. 36 ›› Issue (01): 133-140.DOI: 10.3969/j.issn.1004-132X.2025.01.014

基于深度强化学习的混杂场景目标物体推抓协同策略

胡楷雄¹;宋远航¹;周勇¹*;李卫东²

1.武汉理工大学交通与物流工程学院，武汉，430063
2.上海理工大学机械工程学院，上海，200093

出版日期:2025-01-25 发布日期:2025-03-06
作者简介:胡楷雄，男，1985年生，副教授。研究方向为智能制造。发表论文20余篇。E-mail：kaixiong.hu@whut.edu.cn。
基金资助:
国家自然科学基金(51975444)

A Cooperative Strategy for Pushing and Grasping Target Object in Cluttered Scenes Based on Deep Reinforcement Learning

HU Kaixiong¹;SONG Yuanhang¹;ZHOU Yong¹*;LI Weidong²

1.School of Transportation and Logistics Engineering,Wuhan University of Technology,
Wuhan，430063
2.School of Mechanical Engineering,Shanghai University of Technology,Shanghai，200093

Online:2025-01-25 Published:2025-03-06

摘要/Abstract

摘要： 为提高机器人在混杂场景中抓取被遮挡目标物体的成功率和效率，提出一种基于深度强化学习的“推动”和“抓取”协同推抓策略。该策略利用两个深度Q网络，以RGB-D图像为输入来确定推动或抓取动作，并通过推动改变物体排列以优化抓取条件。该网络使用“抓推抓”三阶段模型训练方法显著提高了抓取能力。基于图像形态处理的方法识别并过滤低质量抓取动作，从而提高成功率和效率。实验结果表明，该方法有效提高了目标物体的抓取成功率和效率。

关键词: 机器人抓取, 混杂场景, 深度强化学习, 多动作协同

Abstract: To improve the success rate and efficiency of robotic grasping for occluded target objects in cluttered scenes, a collaborative push-grasp strategy was proposed based on deep reinforcement learning. The strategy employed 2 deep Q networks and used RGB-D images as inputs to determine push or grasp actions, which optimized object arrangement for better grasping conditions. A “grab-push-grab” three-stage training method was introduced in the model to enhance grasping capabilities significantly. An image morphology-based assessment method effectively identified and filtered low-quality grasp actions to increase successful rates and efficiency. Experimental results confirm that this method significantly enhances the successful rate and efficiency of grasping target objects.

Key words: robot grasping, cluttered scene, deep reinforcement learning, multi-action collaboration

中图分类号:

TG54
TP242.2

胡楷雄1, 宋远航1, 周勇1, 李卫东2. 基于深度强化学习的混杂场景目标物体推抓协同策略[J]. 中国机械工程, 2025, 36(01): 133-140.

HU Kaixiong1, SONG Yuanhang1, ZHOU Yong1, LI Weidong2. A Cooperative Strategy for Pushing and Grasping Target Object in Cluttered Scenes Based on Deep Reinforcement Learning[J]. China Mechanical Engineering, 2025, 36(01): 133-140.

导出引用管理器 EndNote|Ris|BibTeX

链接本文: https://www.cmemo.org.cn/CN/10.3969/j.issn.1004-132X.2025.01.014

https://www.cmemo.org.cn/CN/Y2025/V36/I01/133

参考文献

［1］DUAN Haonan, WANG Peng, HUANG Yayu, et al. Robotics Dexterous Grasping:the Methods Based on Point Cloud and Deep Learning［J］. Frontiers in Neurorobotics, 2021, 15:658280.
［2］BOHG J, MORALES A, ASFOUR T, et al. Data-driven Grasp Synthesis—a Survey［J］. IEEE Transactions on Robotics, 2014, 30(2):289-309.
［3］SAHBANI A, EL-KHOURY S, BIDAUD P. An Overview of 3D Object Grasp Synthesis Algorithms［J］. Robotics and Autonomous Systems, 2012, 60(3):326-336.
［4］LIANG Hongzhuo, MA Xiaojian, LI Shuang, et al. PointNetGPD:Detecting Grasp Configurations from Point Sets［C］∥2019 International Conference on Robotics and Automation(ICRA). Montreal, 2019:3629-3635.
［5］RODRIGUEZ A, MASON M T, FERRY S. From Caging to Grasping［J］. The International Journal of Robotics Research, 2012, 31(7):886-900.
［6］贺辉腾, 周勇, 胡楷雄, 等. 基于深度强化学习的机器人多动作协同抓取策略［J］. 计算机集成制造系统，2024，30（5）：1789-1797.
HE Huiteng, ZHOU Yong, HU Kaixiong, et al. Robot Multi-action Cooperative Grasping Strategy Based on Deep Reinforcement Learning［J］. Computer Integrated Manufacturing Systems，2024，30（5）：1789-1797.
［7］张胜文, 周曦, 李滨城, 等. 基于图像深度学习的零件加工特征信息提取方法［J］. 中国机械工程, 2022, 33(3):348-355.
ZHANG Shengwen, ZHOU Xi, LI Bincheng, et al. Information Extraction Method of Part Machining Features Based on Image Deep Learning［J］. China Mechanical Engineering, 2022, 33(3):348-355.
［8］王勇, 陈荟西, 冯雨齐. 基于改进 CenterNet 的机械臂抓取检测［J］. 中南大学学报(自然科学版), 2021, 52(9):3242-3250.
WANG Yong, CHEN Huixi, FENG Yuqi. Robotic Grasping Detection Based on Improved CenterNet［J］. Journal of Central South University(Science and Technology), 2021, 52(9):3242-3250.
［9］张翰博, 兰旭光, 周欣文, 等. 基于视觉推理的机器人多物体堆叠场景抓取方法［J］. 中国科学(技术科学), 2018, 48(12):1341-1356.
ZHANG Hanbo, LAN Xuguang, ZHOU Xinwen, et al. Robotic Grasping in Multi-object Stacking Scenes Based on Visual Reasoning［J］. Scientia Sinica(Technologica), 2018, 48(12):1341-1356.
［10］孙先涛, 唐思宇, 陈文杰, 等. 复杂环境下基于推抓协同操作的目标物体抓取［J］. 控制理论与应用,2023,40(10):1713-1720.
SUN Xiantao, TANG Siyu, CHEN Wenjie, et al. Target Object Grasp Based on Push-grasp Cooperative Operation in Complex Environment［J］. Control Theory & Applications, 2023,40(10):1713-1720.
［11］李鑫, 沈捷, 曹恺, 等. 深度强化学习的机械臂密集场景多物体抓取方法［J/OL］. 计算机工程与应用，2019:1-9［2023-10-23］.http:∥kns.cnki.net/kcms/detail/11.2127.TP.20231008.1644.008.html.
LI Xin, SHEN Jie, CAO Kai, et al. Deep Reinforcement Learning for Manipulator Multi-object Grasping in Dense Scenes［J/OL］. Computer Engineering and Applications， 2019:1-9［2023-10-23］.http:∥kns.cnki.net/kcms/detail/11.2127.TP.20231008.1644.008.html.
［12］TEN PAS A, PLATT R. Using Geometry to Detect Grasp Poses in 3D Point Clouds［M］∥Robotics Research. Cham:Springer, 2018:307-324.
［13］ZENG A, SONG Shuran, WELKER S, et al. Learning Synergies between Pushing and Grasping with Self-supervised Deep Reinforcement Learning［C］∥2018 IEEE/RSJ International Conference on Intelligent Robots and Systems(IROS). Madrid, 2018:4238-4245.
［14］DENG Yuhong, GUO Xiaofeng, WEI Yixuan, et al. Deep Reinforcement Learning for Robotic Pushing and Picking in Cluttered Environment［C］∥2019 IEEE/RSJ International Conference on Intelligent Robots and Systems(IROS). Macau, 2019:619-626.
［15］TANG Bingjie, CORSARO M, KONIDARIS G, et al. Learning Collaborative Pushing and Grasping Policies in Dense Clutter［C］∥2021 IEEE International Conference on Robotics and Automation(ICRA). Xian, 2021:6177-6184.
［16］YANG Yang, LIANG Hengyue, CHOI C. A Deep Learning Approach to Grasping the Invisible［J］. IEEE Robotics and Automation Letters, 2020, 5(2):2232-2239.
［17］XU Kechun, YU Hongxiang, LAI Qianen, et al. Efficient Learning of Goal-oriented Push-grasping Synergy in Clutter［J］. IEEE Robotics and Automation Letters, 2021, 6(4):6337-6344.
［18］HUANG Gao, LIU Zhuang,van der MAATEN L, et al. Densely Connected Convolutional Networks［C］∥2017 IEEE Conference on Computer Vision and Pattern Recognition(CVPR). Honolulu, 2017:2261-2269.
［19］DENG Jia, DONG Wei, SOCHER R, et al. ImageNet:a Large-scale Hierarchical Image Database［C］∥2009 IEEE Conference on Computer Vision and Pattern Recognition. Miami, 2009:248-255.
［20］NAIR V, HINTON G E. Rectified Linear Units Improve Restricted Boltzmann Machines［C］∥Proceedings of the 27th International Conference on Machine Learning(ICML-10). Haifa，2010:807-814.
［21］IOFFE S, SZEGEDY C. Batch Normalization:Accelerating Deep Network Training by Reducing Internal Covariate Shift［J］. 32nd International Conference on Machine Learning. Lille, 2015:448-456.
［22］KINGMA D P, BA J. Adam:a Method for Stochastic Optimization［EB/OL］. 2014:arXiv:1412.6980. http:∥arxiv.org/abs/1412.6980

[1]	祝正宇1, 郭具涛2, 吕佑龙3, 左丽玲1, 张洁3. 面向柔性作业车间生产调度的深度强化学习方法[J]. 中国机械工程, 2024, 35(11): 2007-2014，2034.
[2]	郭具涛, 吕佑龙, 戴铮, 张洁, 郭宇. 基于复合规则和强化学习的混流装配线调度方法[J]. 中国机械工程, 2023, 34(21): 2600-2606,2614.

基于深度强化学习的混杂场景目标物体推抓协同策略

A Cooperative Strategy for Pushing and Grasping Target Object in Cluttered Scenes Based on Deep Reinforcement Learning

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 2

编辑推荐

Metrics