二阶锥约束规划的机器人视觉闭环位姿自协调方法

doi:10.3969/j.issn.1004-132X.2024.06.012

中国机械工程 ›› 2024, Vol. 35 ›› Issue (06): 1064-1073.DOI: 10.3969/j.issn.1004-132X.2024.06.012

二阶锥约束规划的机器人视觉闭环位姿自协调方法

仲训杲^1,4;罗家国¹;田军²;仲训昱²;彭侠夫²;刘强³

1.厦门理工学院电气工程与自动化学院，厦门，361024
2.厦门大学航空航天学院，厦门，361005
3.布里斯托大学工程数学与技术学院,布里斯托,BS8 1TW
4.厦门市高端电力装备及智能控制重点实验室，厦门，361024

出版日期:2024-06-25 发布日期:2024-07-30
作者简介:仲训杲，男，1983年生，副教授。研究方向为机器人视觉伺服、机器人智能化等。E-mail:zhongxungao@163.com。
基金资助:
国家自然科学基金（61703356）；福建省自然科学基金（2022J011256）

Robot Vision Closed-loop Pose Autonomous Coordination Method with Second-order Cone Constrained Programming

ZHONG Xungao1,4;LUO Jiaguo1;TIAN Jun2;ZHONG Xunyu2;PENG Xiafu2;LIU Qiang3

1.School of Electrical Enginnering and Automation, Xiamen University of Technology, Xiamen,
Fujian,361024
2.School of Aerospace Engineering,Xiamen University,Xiamen,Fujian,361002
3.School of Engineering Mathematics and Technology,University of Bristol,Bristol,BS8 1TW
4.Xiamen Key Laboratory of Frontier Electric Power Equipment and Intelligent Control,Xiamen,
Fujian,361024

Online:2024-06-25 Published:2024-07-30

摘要/Abstract

摘要： 将机器人“手眼”位姿自协调视为无标定约束规划问题,提出一种基于二阶锥约束规划的视觉闭环控制方法。在图像平面和笛卡儿空间分别建立基于图像与基于位姿的视觉伺服控制算法;在路径约束和局部极小约束准则下构建二阶锥凸优化模型,实现了图像特征轨迹和机器人运动路径折中最优;将所提出的二阶锥约束规划模型嵌入自适应状态估计器，实现了机器人雅可比矩阵在线映射学习,解决了“手眼”标定参数和视觉深度信息未知问题；最后通过无标定机器人视觉定位证明了二阶锥约束规划模型的有效性,真实环境抓取操作表明了机器人位姿自协调控制的可行性。

关键词: 位姿协调, 约束规划, 无标定视觉伺服, 混合闭环反馈控制

Abstract: A robot “hand-eye” pose autonomous coordination was regarded as a uncalibration constrained programming problem, and a visual closed-loop control method was proposed based on second-order cone constrained programming. Firstly, the visual servoing control algorithms were constructed in the image planes and Cartesian space, respectively based on images and positions. After that, by established the path constraint and the local minimal constraint rules, and a second-order cone convex optimization model was constructed to realize the compromise optimal control of image feature trajectory and robot motion path. Moreover, the proposed second-order cone constrained programming model was embedded with an adaptive state estimator, to realize robotic Jacobian matrix online mapping learning, and to solve the unknown problems of “hand-eye” calibration parameters and visual depth information. Finally, the uncalibrated robot visual positioning experiments prove the effectiveness of the convex optimization planning model, and the real grasping tasks illustrate the feasibility of the robot pose autonomous coordination.

Key words: pose coordination, constrained programming, uncalibration visual servoing, hybrid closed-loop feedback control

中图分类号:

TP242.2

仲训杲1, 4, 罗家国1, 田军2, 仲训昱2, 彭侠夫2, 刘强3. 二阶锥约束规划的机器人视觉闭环位姿自协调方法[J]. 中国机械工程, 2024, 35(06): 1064-1073.

ZHONG Xungao1, 4, LUO Jiaguo1, TIAN Jun2, ZHONG Xunyu2, PENG Xiafu2, LIU Qiang3. Robot Vision Closed-loop Pose Autonomous Coordination Method with Second-order Cone Constrained Programming[J]. China Mechanical Engineering, 2024, 35(06): 1064-1073.

导出引用管理器 EndNote|Ris|BibTeX

链接本文: http://www.cmemo.org.cn/CN/10.3969/j.issn.1004-132X.2024.06.012

http://www.cmemo.org.cn/CN/Y2024/V35/I06/1064

参考文献

［1］LI T, YU J P, QIU Q. Hybrid Uncalibrated Visual Servoing Control of Harvesting Robots with RGB-D Cameras［J］. IEEE Transactions on Industrial Electronics, 2023, 70(3):2729-2737.
［2］MARWAN Q M, KWEK L C, CHUA S C. Review of Deep Reinforcement Learning-based Object Grasping:Techniques, Open Challenges, and Recommendations［J］. IEEE Access, 2020, 8:178450-178481.
［3］LI R, QIAO H. A Survey of Methods and Strategies for High-precision Robotic Grasping and Assembly, Tasks-some New Trends［J］. IEEE/ASME Transactions on Mechatronics, 2019, 24(6):2718-2732.
［4］SUN F, LIU C, HUANG W, et al. Object Classification and Grasp Planning Using Visual and Tactile Sensing［J］. IEEE Transactions on Systems, Man, and Cybernetics:Systems, 2016,46(7):969-979.
［5］GOU M, FAND H S, ZHU Z, et al. RGB Matters:Learning 7-DoF Grasp Poses on Monocular RGBD Images［C］∥Proceeding of the 2021 IEEE International Conference on Robotics Automation(ICRA). Xian:IEEE, 2021:1-8.
［6］仲训杲, 徐敏, 仲训昱, 等. 基于多模特征深度学习的机器人抓取判别方法［J］. 自动化学报, 2016, 42(7):1022-1029.
ZHONG Xungao, XU Min, ZHONG Xunyu,et al. Multimodal Features Deep Learning for Robotic Potential Grasp Recognition［J］. Acta Automatica Sinica, 2016, 42(7):1022-1029.
［7］MAHLER J，MATL M，SATISH V，et al. Learning Ambidextrous Robot Grasping Policies［J］. Science Robotics, 2019, 26(4):1-12.
［8］黄艳龙, 徐德, 谭民. 机器人运动轨迹的模仿学习综述［J］. 自动化学报, 2022, 48(2):315-334.
HUANG Yanlong, XU De, TAN Min. On Imitation Learning of Robot Movement Trajectories:a Survey［J］. Acta Automatica Sinica, 2022, 48(2):315-334.
［9］SHIVANI K, SAMHITA M, BENJAMIN W, et al. Visual Servoing for Pose Control of Soft Continuum Arm in a Structured Environment［J］. IEEE Robotics and Automation Letters, 2022, 7(2):5504-5511.
［10］LEE C, SEO H, KIM J. Position-based Monocular Visual Servoing of an Unknown Target Using Online Self-supervised Learning［C］∥Proceeding of the 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems(IROS). Macau:IEEE, 2019:4467-4473.
［11］PARK D H， KWON J H， HA I J. Novel Position-based Visual Servoing Approach to Robust Global Stability Under Field-of-view Constraint［J］. IEEE Transactions on Industrial Electronics, 2012, 59(12):4735-4752.
［12］NICOLAS G A, CARLOS P V,JOSE M S ,et al. Robust and Cooperative Image-based Visual Servoing System Using a Redundant Architecture［J］. Sensors, 2011, 11(12):11885-11900.
［13］杨唐文, 高立宁, 阮秋琦,等. 移动双臂机械手系统协调操作的视觉伺服技术［J］. 控制理论与应用, 2015, 32(1):69-74.
YAMG Tangwen, GAO Lining, RUAN Qiuqi, et al. Visual Servo Technology for Coordinated Manipulation of a Mobile Dual-arm Manipulator System［J］. Control Theory & Applications, 2015, 32(1):69-74.
［14］REDWAN D, NICOLAS A, YOUCEF M, et al. Dynamic Visual Servoing from Sequential Regions of Interest Acquisition［J］. The International Journal of Robotics Research, 2012, 31(4):1-19.
［15］LI J, XIE H, LOW K H, et al. Image-based Visual Servoing of Rotorcrafts to Planar Visual Targets of Arbitrary Orientation［J］. IEEE Robotics and Automation Letters, 2021, 6(4):7861-7868.
［16］李宝全, 方勇纯, 张雪波. 基于2D三焦点张量的移动机器人视觉伺服镇定控制［J］. 自动化学报, 2014, 40(12):2706-2715.
LI Baoquan, FANG Yongchun, ZHANG Xuebo. 2D Trifocal Tensor Based Visual Servo Regulation of Nonholonomic Mobile Robots［J］. Acta Automatica Sinica, 2014, 40(12):2706-2715.
［17］徐璠, 王贺升. 软体机械臂水下自适应鲁棒视觉伺服［J］. 自动化学报, 2023, 49(4):744-753.
XU Fang, WANG Hesheng. Adaptive Robust Visual Servoing Control of a Soft Manipulator in Underwater Environment［J］. Acta Automatica Sinica, 2023, 49(4):744-753.
［18］WANG H S, JIANG M K, CHEN W D, et al. Visual Servoing of Robots with Uncalibrated Robot and Camera Parameters［J］. Mechatronics, 2012, 22(7):661-668.
［19］LIANG X W, WANG H S, CHEN W D, et al. Adaptive Image-based Trajectory Tracking Control of Wheeled Mobile Robots with an Uncalibrated Fixed Camera［J］. IEEE Transactions on Control Systems Technology, 2015, 23(6):2266-2282.
［20］HAO M, SUN Z. A Universal State-space Approach to Uncalibrated Model-free Visual Servoing［J］. IEEE/ASME Transactions on Mechatronics, 2012, 17(5):833-846.
［21］CHESI G, HUNG Y S. Global Path-planning for Constrained and Optimal Visual Servoing［J］. IEEE Transactions on Robotics, 2007, 23(5):1050-1060.
［22］MARCO C, GIUSEPPE D M, CIRO N, et al.Stability and Convergence Analysis of 3D Feature-based Visual Servoing［J］. IEEE Robotics and Automation Letters, 2022, 7(4)：12022-12019.

[1]	张铭鑫, 葛茂根, 张玺, 刘从虎, 凌琳, 扈静. 基于模糊机会约束规划的再制造装配车间调度优化方法[J]. 中国机械工程, 2015, 26(11): 1488-1493.
[2]	王淑旺, 刘志峰, 胡迪, 高洋, 蒋浩. 基于不确定规划的产品拆卸序列规划[J]. 中国机械工程, 2012, 23(18): 2143-2148.
[3]	付清山;张志胜;史金飞;. 一种采用L-M方法的无标定视觉伺服控制方法[J]. J4, 2008, 19(21): 0-2645.

二阶锥约束规划的机器人视觉闭环位姿自协调方法

Robot Vision Closed-loop Pose Autonomous Coordination Method with Second-order Cone Constrained Programming

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 3

编辑推荐

Metrics