Gesture-based human-robot interface for dual-robot with hybrid sensors

Abstract

Purpose

The purpose of this paper is the research of a novel gesture-based dual-robot collaborative interaction interface, which achieves the gesture recognition when both hands overlap. This paper designs a hybrid-sensor gesture recognition platform to detect the both-hand data for dual-robot control.

Design/methodology/approach

This paper uses a combination of Leap Motion and PrimeSense in the vertical direction, which detects both-hand data in real time. When there is occlusion between hands, each hand is detected by one of the sensors, and a quaternion-based algorithm is used to realize the conversion of two sensors corresponding to different coordinate systems. When there is no occlusion, the data are fused by a self-adaptive weight fusion algorithm. Then the collision detection algorithm is used to detect the collision between robots to ensure safety. Finally, the data are transmitted to the dual robots.

Findings

This interface is implemented on a dual-robot system consisting of two 6-DOF robots. The dual-robot cooperative experiment indicates that the proposed interface is feasible and effective, and it takes less time to operate and has higher interaction efficiency.

Originality/value

A novel gesture-based dual-robot collaborative interface is proposed. It overcomes the problem of gesture occlusion in two-hand interaction with low computational complexity and low equipment cost. The proposed interface can perform a long-term stable tracking of the two-hand gestures even if there is occlusion between the hands. Meanwhile, it reduces the number of hand reset to reduce the operation time. The proposed interface achieves a natural and safe interaction between the human and the dual robot.

Citation

Zhang, B., Du, G., Shen, W. and Li, F. (2019), "Gesture-based human-robot interface for dual-robot with hybrid sensors", Industrial Robot, Vol. 46 No. 6, pp. 800-811. https://doi.org/10.1108/IR-11-2018-0245