Search results

Autonomous robots must be able to understand long-term manipulation tasks described by humans and perform task analysis and planning based on the current environment in a variety of scenes, such as daily manipulation and industrial assembly. However, both classical task and motion planning algorithms and single data-driven learning planning methods have limitations in practicability, generalization and interpretability. The purpose of this work is to overcome the limitations of the above methods and achieve generalized and explicable long-term robot manipulation task planning.

The authors propose a planning method for long-term manipulation tasks that combines the advantages of existing methods and the prior cognition brought by the knowledge graph. This method integrates visual semantic understanding based on scene graph generation, regression planning based on deep learning and multi-level representation and updating based on a knowledge base.

The authors evaluated the capability of this method in a kitchen cooking task and tabletop arrangement task in simulation and real-world environments. Experimental results show that the proposed method has a significantly improved success rate compared with the baselines and has excellent generalization performance for new tasks.

The authors demonstrate that their method is scalable to long-term manipulation tasks with varying complexity and visibility. This advantage allows their method to perform better in new manipulation tasks. The planning method proposed in this work is meaningful for the present robot manipulation task and can be intuitive for similar high-level robot planning.

For robots to more actively interact with the surrounding environment in object manipulation tasks or walking, they must understand the physical attributes of objects and surface materials they encounter. Dynamic tactile sensing can effectively capture rich information about material properties. Hence, methods that convey and interpret this tactile information to the user can improve the quality of human–machine interaction. This paper aims to propose a visual-tactile cross-modal retrieval framework to convey tactile information of surface material for perceptual estimation.

The tactile information of a new unknown surface material can be used to retrieve perceptually similar surface from an available surface visual sample set by associating tactile information to visual information of material surfaces. For the proposed framework, the authors propose an online low-rank similarity learning method, which can effectively and efficiently capture the cross-modal relative similarity between visual and tactile modalities.

Experimental results conducted on the Technischen Universität München Haptic Texture Database demonstrate the effectiveness of the proposed framework and the method.

This paper provides a visual-tactile cross-modal perception method for recognizing material surface. By the method, a robot can communicate and interpret the conveyed information about the surface material properties to the user; it will further improve the quality of robot interaction.

The purpose of the paper is to present a novel cross-modal sensor whose tactile is computed by the visual information. The proposed sensor can measure the forces of robotic grasping.

The proposed cross-modal tactile sensor consists of a transparent elastomer with markers, a camera, an LED circuit board and supporting structures. The model and performance of the elastomer are analyzed. Then marker recognition method is proposed to determine the movements of the marker on the surface, and the force calculation algorithm is presented to compute the three-dimension force.

Experimental results demonstrate that the proposed tactile sensor can accurately measure robotic grasping forces.

The proposed cross-modal tactile sensor determines the robotic grasping forces by the images of markers. It can give more information of the force than traditional tactile sensors. Meanwhile, the proposed algorithms for forces calculation determine the superior results.

The purpose of this paper is to present a novel tactile sensor and a visual-tactile recognition framework to reduce the uncertainty of the visual recognition of transparent objects.

A multitask learning model is used to recognize intuitive appearance attributes except texture in the visual mode. Tactile mode adopts a novel vision-based tactile sensor via the level-regional feature extraction network (LRFE-Net) recognition framework to acquire high-resolution texture information and temperature information. Finally, the attribute results of the two modes are integrated based on integration rules.

The recognition accuracy of attributes, such as style, handle, transparency and temperature, is near 100%, and the texture recognition accuracy is 98.75%. The experimental results demonstrate that the proposed framework with a vision-based tactile sensor can improve attribute recognition.

Transparency and visual differences make the texture of transparent glass hard to recognize. Vision-based tactile sensors can improve the texture recognition effect and acquire additional attributes. Integrating visual and tactile information is beneficial to acquiring complete attribute features.

This study aims to process multi-agent system with kinds of limitations and constraints, and consider the robot in-hand manipulation as a problem of coordination and cooperation of multi-fingered hand.

A cooperative distributed model predictive control (MPC) algorithm is proposed to perform robot in-hand manipulation.

A cooperative distributed MPC approach is formulated for robot in-hand manipulation problem, which enables address complex limitation and constraint conditions in object motion planning, and realizes tracking trajectory of the object more than tracking position of the object.

This method to implement the moving object task uses the kinematic parameters without the knowledge of dynamic properties of the object. The cooperative distributed MPC scheme is designed to guarantee the movement of the object to a desired position and trajectory at algorithmic level.

The purpose of this paper is to present a novel data glove which can capture the motion of the arm and hand by inertial and magnetic sensors. The proposed data glove is used to provide the information of the gestures and teleoperate the robotic arm-hand.

The data glove comprises 18 low-cost inertial and magnetic measurement units (IMMUs) which not only make up the drawbacks of traditional data glove that only captures the incomplete gesture information but also provide a novel scheme of the robotic arm-hand teleoperation. The IMMUs are compact and small enough to wear on the upper arm, forearm, palm and fingers. The calibration method is proposed to improve the accuracy of measurements of units, and the orientations of each IMMU are estimated by a two-step optimal filter. The kinematic models of the arm, hand and fingers are integrated into the entire system to capture the motion gesture. A positon algorithm is also deduced to compute the positions of fingertips. With the proposed data glove, the robotic arm-hand can be teleoperated by the human arm, palm and fingers, thus establishing a novel robotic arm-hand teleoperation scheme.

Experimental results show that the proposed data glove can accurately and fully capture the fine gesture. Using the proposed data glove as the multiple input device has also proved to be a suitable teleoperating robotic arm-hand system.

Integrated with 18 low-cost and miniature IMMUs, the proposed data glove can give more information of the gesture than existing devices. Meanwhile, the proposed algorithms for motion capture determine the superior results. Furthermore, the accurately captured gestures can efficiently facilitate a novel teleoperation scheme to teleoperate the robotic arm-hand.

This paper aims to propose a novel active SLAM framework to realize avoid obstacles and finish the autonomous navigation in indoor environment.

The improved fuzzy optimized Q-Learning (FOQL) algorithm is used to solve the avoidance obstacles problem of the robot in the environment. To reduce the motion deviation of the robot, fractional controller is designed. The localization of the robot is based on FastSLAM algorithm.

Simulation results of avoiding obstacles using traditional Q-learning algorithm, optimized Q-learning algorithm and FOQL algorithm are compared. The simulation results show that the improved FOQL algorithm has a faster learning speed than other two algorithms. To verify the simulation result, the FOQL algorithm is implemented on a NAO robot and the experimental results demonstrate that the improved fuzzy optimized Q-Learning obstacle avoidance algorithm is feasible and effective.

The improved fuzzy optimized Q-Learning (FOQL) algorithm is used to solve the avoidance obstacles problem of the robot in the environment. To reduce the motion deviation of the robot, fractional controller is designed. To verify the simulation result, the FOQL algorithm is implemented on a NAO robot and the experimental results demonstrate that the improved fuzzy optimized Q-Learning obstacle avoidance algorithm is feasible and effective.

The authors want to design an adaptive grasping control strategy without setting the expected contact force in advance to maintain grasping stable, so that the proposed control system can deal with unknown object grasping manipulation tasks.

The adaptive grasping control strategy is proposed based on bang-bang-like control principle and slippage detection module. The bang-bang-like control method is designed to find and set the expected contact force for the whole control system, and the slippage detection function is achieved by dynamic time warping algorithm.

The expected contact force can adaptively adjust in grasping tasks to avoid bad effects on the control system by the differences of prior test results or designers. Slippage detection can be recognized in time with variation of expected contact force manipulation environment in the control system. Based on if the slippage caused by an unexpected disturbance happens, the control system can automatically adjust the expected contact force back to the level of the previous stable state after a given time, and has the ability to identify an unnecessary increasing in the expected contact force.

Only contact force is used as feedback variable in control system, and the proposed strategy can save hardware components and electronic circuit components for sensing, reducing the cost and design difficulty of conducting real control system and making it easy to realize in engineering application field. The expected contact force can adaptively adjust due to unknown disturbance and slippage for various grasping manipulation tasks.

This study aims to focus on the analysis of the internal mechanism of farmers’ ecological cognition and the behaviour of Grain for Green Project (GGP), and the further relationship between ecological cognition and ecological aspiration, proposing climate change strategies and management from the perspective of farmers.

Theory of planned behaviour and social exchange theory were used to construct a theoretical framework and an ecological cognition under the influence of external factors, the aspiration and the behaviour of GGP, using ecological fragile areas in Bazhou and Changji, Xinjiang of 618 peasant households’ survey data. The structural equation model and Heckman two-step model were applied to analyse the relationship between ecological cognition and ecological aspiration of farmers, the impact of peasant households’ ecological cognition and aspiration to the behaviour of GGP and the influence factors of GGP behaviour.

This research’s results show that the three characterizations of ecological cognitive variables, attitude towards the behaviour (AB), subjective norms (SN) and perceived behaviour control (PBC), have significant positive impact on farmers’ GGP ecological aspiration. The comprehensive impact path coefficients of ecological cognition are PBC (0.498) > SN (0.223) > AB (0.177). Also, income change is a moderating variable, which has a significant moderating effect on the influence of AB and SN on ecological aspiration. Further, farmers’ ecological cognition has an influence on the behaviour of GGP, and the change of farmers’ income has a significant positive effect on farmers’ choice of returning farmland to forests.

The ecological protection policy suggestions and countermeasures can be drawn from the research conclusions, adapted to China’s ecologically fragile regions and even similar regions in the world to response the climate change.

Combining the theory of planning behaviour and social exchange, this paper empirically analyses the path of farmers’ ecological cognition and ecological aspiration, as well as the influencing factors.