A New Visual Servoing Method for Grasping and Assembling Objects using Stereo Image Based Feedback

Subject Areas : robotics

mahmoud jeddi ¹ , ahmad reza khoogar ²

1 - Faculty of Material and Manufacturing Technologies Malek Ashtar University of Technology, Tehran, Iran
2 - Faculty of Material and Manufacturing Technologies Malek Ashtar University of Technology, Tehran, Iran

Received: 2021-02-25 Accepted : 2021-06-17 Published : 2023-03-01

Keywords: Stereo Image Based Visual Servoing, Image interaction matrix, Recursive Least Square, Feature matching, 6 DOF PILZ robot,

Abstract :

In this paper, an eye-in-hand stereo image-based visual serving controller for industrial 6 degrees of freedom manipulator robots is presented. The visual control algorithms mostly use the relationship between camera speed and changes in image features, to determine the end-effector movement path. One of the main problems of the classical IBVS method is the inability to estimate the distance of the object related to the camera, which requires peripheral equipment such as a laser rangefinder to estimate the depth. In this study, two cameras were mounted on the end-effector of a 6 DOF manipulator robot. The distance of the object to the camera is estimated by the equations associated with the epipolar plane, and the interaction matrix is updated at any time. For increasing response speed, the image interaction matrix was divided into two separate parts related to translational and rotational motion, and it was found that only the translational motion part is affected by distance. The control method separates the camera motion into three-stage based on pure rotation, pure translation, and hybrid motion, which has a better time response compared to the classical IBVS control methods. Additionally, a method for position prediction and trajectory estimation of the moving target in order to use in a real-time grasping task is proposed and developed using Recursive Least Square as the trajectory estimators in the image plane. The simulation results show that the proposed method increases the system response speed and improves the tracking performance.

References:

• Hashimoto, K., A review on vision-based control of robot manipulators, Adv. Robot. Int. J. Robot. Soc. Japan, Vol. 17, No. 10, 2003, pp. 969–991.


• Corke, P., Robotics, Vision and Control: Fundamental Algorithms In MATLAB®Second, Completely Revised, Vol. 118, 2017.


• Chaumette, F., Hutchinson, S., Visual Servo Control, I. Basic Approaches, IEEE Robot. Autom. Mag., Vol. 13, No. 4, 2006, pp. 82–90.


• Chaumette, F., Hutchinson, S., Visual Servo Control, II. Advanced Approaches [Tutorial], IEEE Robot. Autom. Mag., Vol. 14, No. 1, 2007, pp. 109–118.


• Keshmiri, M., Xie, W. F., and Mohebbi, A., Augmented Image-Based Visual Servoing of A Manipulator Using Acceleration Command, IEEE Trans. Ind. Electron., Vol. 61, No. 10, 2014, pp. 5444–5452.


• Chaumette, F., Potential Problems of Stability and Convergence in Image-Based and Position-Based Visual Servoing, In The Confluence of Vision and Control, Springer, 1998, pp. 66–78.


• Xie, W. F., Li, Z., Tu, X. W., and Perron, C., Switching Control of Image-Based Visual Servoing With Laser Pointer in Robotic Manufacturing Systems, IEEE Trans. Ind. Electron., Vol. 56, No. 2, 2008, pp. 520–529.


• Shu, T., Gharaaty, S., Xie, W., Joubair, A., and Bonev, I. A., Dynamic Path Tracking of Industrial Robots with High Accuracy Using Photogrammetry Sensor, IEEE/ASME Trans. Mechatronics, Vol. 23, No. 3, 2018, pp. 1159–1170.


• Shen, Y., Xiang, G., Liu, Y. H., and Li, K., Uncalibrated Visual Servoing of Planar Robots, In Proceedings 2002 IEEE International Conference on Robotics and Automation (Cat. No. 02CH37292), Vol. 1, 2002, pp. 580–585.


• Ghasemi, A., Li, P., and Xie, W. F., Adaptive Switch Image-Based Visual Servoing for Industrial Robots, Int. J. Control. Autom. Syst., Vol. 18, No. 5, 2020, pp. 1324–1334.


• Ghasemi A., Xie, W. F., Decoupled Image-Based Visual Servoing for Robotic Manufacturing Systems Using Gain Scheduled Switch Control, In 2017 International Conference on Advanced Mechatronic Systems (ICAMechS), 2017, pp. 94–99.


• Liang, X., Wang, H., Liu, Y. H., Chen, W., and Zhao, J., A Unified Design Method for Adaptive Visual Tracking Control of Robots with Eye-In-Hand/Fixed Camera Configuration, Automatica, Vol. 59, 2015, pp. 97–105.


• Wang, H., Adaptive Visual Tracking for Robotic Systems Without Image-Space Velocity Measurement, Automatica, Vol. 55, 2015, pp. 294–301.


• Dong G., Zhu, Z. H., Kinematics-Based Incremental Visual Servo for Robotic Capture of Non-Cooperative Target, Rob. Auton. Syst., Vol. 112, 2019, pp. 221–228.


• Keshmiri M., Xie, W. F., Image-Based Visual Servoing Using an Optimized Trajectory Planning Technique, IEEE/ASME Trans. Mechatronics, Vol. 22, No. 1, 2016, pp. 359–370.


• Zheng, D., Wang, H., Chen, W., and Wang, Y., Planning and Tracking in Image Space for Image-Based Visual Servoing of A Quadrotor, IEEE Trans. Ind. Electron., Vol. 65, No. 4, 2017, pp. 3376–3385.


• Zhao, Y., Xie, W. F., and Liu, S., Image-Based Visual Servoing Using Improved Image Moments in 6-DOF Robot Systems, Int. J. Control. Autom. Syst., Vol. 11, No. 3, 2013, pp. 586–596.


• Payeur, P., Le-Huy, H., and Gosselin, C. M., Trajectory Prediction for Moving Objects Using Artificial Neural Networks, IEEE Trans. Ind. Electron., Vol. 42, No. 2, pp. 147–158, 1995.


• Hashimoto, H., Kubota, T., Sato, M., and Harashima, F., Visual Control of Robotic Manipulator Based on Neural Networks, IEEE Trans. Ind. Electron., Vol. 39, No. 6, pp. 490–496, 1992.


• Martinet, P., Cervera, E., Stacking Jacobians Properly in Stereo Visual Servoing, In Proceedings 2001 ICRA. IEEE International Conference on Robotics and Automation (Cat. No. 01CH37164), Vol. 1, 2001, pp. 717–722.


• Kim, D. J., Lovelett, R., and Behal, A., Eye-in-Hand Stereo Visual Servoing of an Assistive Robot Arm in Unstructured Environments, In 2009 IEEE International Conference on Robotics and Automation, 2009, pp. 2326–2331.