Search results

Scattered parts are laid randomly during the manufacturing process and have difficulty to recognize and manipulate. This study aims to complete the grasp of the scattered parts by a manipulator with a camera and learning method.

In this paper, a cascaded convolutional neural network (CNN) method for robotic grasping based on monocular vision and small data set of scattered parts is proposed. This method can be divided into three steps: object detection, monocular depth estimation and keypoint estimation. In the first stage, an object detection network is improved to effectively locate the candidate parts. Then, it contains a neural network structure and corresponding training method to learn and reason high-resolution input images to obtain depth estimation. The keypoint estimation in the third step is expressed as a cumulative form of multi-scale prediction from a network to use an red green blue depth (RGBD) map that is acquired from the object detection and depth map estimation. Finally, a grasping strategy is studied to achieve successful and continuous grasping. In the experiments, different workpieces are used to validate the proposed method. The best grasping success rate is more than 80%.

By using the CNN-based method to extract the key points of the scattered parts and calculating the possibility of grasp, the successful rate is increased.

This method and robotic systems can be used in picking and placing of most industrial automatic manufacturing or assembly processes.

Unlike standard parts, scattered parts are randomly laid and have difficulty recognizing and grasping for the robot. This study uses a cascaded CNN network to extract the keypoints of the scattered parts, which are also labeled with the possibility of successful grasping. Experiments are conducted to demonstrate the grasping of those scattered parts.

This paper aims to propose a new solution for real-time 3D perception with monocular camera. Most of the industrial robots’ solutions use active sensors to acquire 3D structure information, which limit their applications to indoor scenarios. By only using monocular camera, some state of art method provides up-to-scale 3D structure information, but scale information of corresponding objects is uncertain.

First, high-accuracy and scale-informed camera pose and sparse 3D map are provided by leveraging ORB-SLAM and marker. Second, for each frame captured by a camera, a specially designed depth estimation pipeline is used to compute corresponding 3D structure called depth map in real-time. Finally, depth map is integrated into volumetric scene model. A feedback module has been designed for users to visualize intermediate scene surface in real-time.

The system provides more robust tracking performance and compelling results. The implementation runs near 25 Hz on mainstream laptop based on parallel computation technique.

A new solution for 3D perception is using monocular camera by leveraging ORB-SLAM systems. Results in our system are visually comparable to active sensor systems such as elastic fusion in small scenes. The system is also both efficient and easy to implement, and algorithms and specific configurations involved are introduced in detail.

Single image depth prediction allows to extract depth information from a usual 2D image without usage of special sensors such as laser sensors, stereo cameras, etc. The purpose of this paper is to solve the problem of obtaining depth information from 2D image by applying deep neural networks (DNNs).

Several experiments and topologies are presented: DNN that uses three inputs—sequence of 2D images from videostream and DNN that uses only one input. However, there is no data set, that contains videostream and corresponding depth maps for every frame. So technique of creating data sets using the Blender software is presented in this work.

Despite the problem of an insufficient amount of available data sets, the problem of overfitting was encountered. Although created models work on the data sets, they are still overfitted and cannot predict correct depth map for the random images, that were included into the data sets.

Existing techniques of depth images creation are tested, using DNN.

The interdisciplinary nature of video networking, coupled with various recent developments in standards, proposals and applications, poses great challenges to the research and industrial communities working in this area. The main purpose of this paper is to provide a tutorial and survey on recent advances in video networking from an integrated perspective of both video signal processing and networking.

Detailed technical descriptions and insightful analysis are presented for recent and emerging video coding standards, in particular the H.264 family. The applications of selected video coding standards in emerging wireless networks are then introduced with an emphasis on scalable video streaming in multihomed mobile networks. Both research challenges and potential solutions are discussed along the description, and numerical results through simulations or experiments are provided to reveal the performances of selected coding standards and networking algorithms.

The tutorial helps to clarify the similarities and differences among the considered standards and networking applications. A number of research trends and challenges are identified, and selected promising solutions are discussed. This practice would provoke further thoughts on the development of this area and open up more research and application opportunities.

Not all the concerned video coding standards are complemented with thorough studies of networking application scenarios.

The discussed video coding standards are either playing or going to play indispensable roles in the video industry; the introduced networking scenarios bring together these standards and various emerging wireless networking paradigms towards innovative application scenarios.

The comprehensive overview and critiques on existing standards and application approaches offer a valuable reference for researchers and system developers in related research and industrial communities.

Aiming at the problem that quadruped crawling robot is easy to collide and overturn when facing obstacles and bulges in the process of complex slope movement, this paper aims to propose an obstacle avoidance gait planning of quadruped crawling robot based on slope terrain recognition.

First, considering the problem of low uniformity of feature points in terrain recognition images under complex slopes, which leads to too long feature point extraction time, an improved ORB (Oriented FAST and Rotated BRIEF) feature point extraction method is proposed; second, when the robot avoids obstacles or climbs over bumps, aiming at the problem that the robustness of a single step cannot satisfy the above two motions at the same time, the crawling gait is planned according to the complex slope terrain, and a robot obstacle avoidance gait planning based on the artificial potential field method is proposed. Finally, the slope walking experiment is carried out in the Robot Operating System.

The proposed method provides a solution for the efficient walking of robot under slope. The experimental results show that the extraction time of the improved ORB extraction algorithm is 12.61% less than the original ORB extraction algorithm. The vibration amplitude of the robot’s centroid motion curve is significantly reduced, and the contact force is reduced by 7.76%. The time it takes for the foot contact force to stabilize has been shortened by 0.25 s. This fact is verified by simulation and test.

The method proposed in this paper uses the improved feature point recognition algorithm and obstacle avoidance gait planning to realize the efficient walking of quadruped crawling robot on the slope. The walking stability of quadruped crawling robot is tested by prototype.

The purpose of this paper is to propose a multi-feature, multi-metric and multi-loop tightly coupled LiDAR-visual-inertial odometry, M³LVI, for high-accuracy and robust state estimation and mapping.

M³LVI is built atop a factor graph and composed of two subsystems, a LiDAR-inertial system (LIS) and a visual-inertial system (VIS). LIS implements multi-feature extraction on point cloud, and then multi-metric transformation estimation is implemented to realize LiDAR odometry. LiDAR-enhanced images and IMU pre-integration have been used in VIS to realize visual odometry, providing a reliable initial guess for LIS matching module. Location recognition is performed by a dual loop module combined with Bag of Words and LiDAR-Iris to correct accumulated drift. M³LVI also functions properly when one of the subsystems failed, which greatly increases the robustness in degraded environments.

Quantitative experiments were conducted on the KITTI data set and the campus data set to evaluate the M³LVI. The experimental results show the algorithm has higher pose estimation accuracy than existing methods.

The proposed method can greatly improve the positioning and mapping accuracy of AGV, and has an important impact on AGV material distribution, which is one of the most important applications of industrial robots.

M³LVI divides the original point cloud into six types, and uses multi-metric transformation estimation to estimate the state of robot and adopts factor graph optimization model to optimize the state estimation, which improves the accuracy of pose estimation. When one subsystem fails, the other system can complete the positioning work independently, which greatly increases the robustness in degraded environments.

Simultaneous localization and map building (SLAM), as a state estimation problem, is a prerequisite for solving the problem of autonomous vehicle motion in unknown environments. Existing algorithms are based on laser or visual odometry; however, the lidar sensing range is small, the amount of data features is small, the camera is vulnerable to external conditions and the localization and map building cannot be performed stably and accurately using a single sensor. This paper aims to propose a laser three dimensions tightly coupled map building method that incorporates visual information, and uses laser point cloud information and image information to complement each other to improve the overall performance of the algorithm.

The visual feature points are first matched at the front end of the method, and the mismatched point pairs are removed using the bidirectional random sample consensus (RANSAC) algorithm. The laser point cloud is then used to obtain its depth information, while the two types of feature points are fed into the pose estimation module for a tightly coupled local bundle adjustment solution using a heuristic simulated annealing algorithm. Finally, the visual bag-of-words model is fused in the laser point cloud information to establish a threshold to construct a loopback framework to further reduce the cumulative drift error of the system over time.

Experiments on publicly available data sets show that the proposed method in this paper can match its real trajectory well. For various scenes, the map can be constructed by using the complementary laser and vision sensors, with high accuracy and robustness. At the same time, the method is verified in a real environment using an autonomous walking acquisition platform, and the system loaded with the method can run well for a long time and take into account the environmental adaptability of multiple scenes.

A multi-sensor data tight coupling method is proposed to fuse laser and vision information for optimal solution of the positional attitude. A bidirectional RANSAC algorithm is used for the removal of visual mismatched point pairs. Further, oriented fast and rotated brief feature points are used to build a bag-of-words model and construct a real-time loopback framework to reduce error accumulation. According to the experimental validation results, the accuracy and robustness of the single-sensor SLAM algorithm can be improved.

This paper primarily aims to focus on a review of convolutional neural network (CNN)-based eye control systems. The performance of CNNs in big data has led to the development of eye control systems. Therefore, a review of eye control systems based on CNNs is helpful for future research.

In this paper, first, it covers the fundamentals of the eye control system as well as the fundamentals of CNNs. Second, the standard CNN model and the target detection model are summarized. The eye control system’s CNN gaze estimation approach and model are next described and summarized. Finally, the progress of the gaze estimation of the eye control system is discussed and anticipated.

The eye control system accomplishes the control effect using gaze estimation technology, which focuses on the features and information of the eyeball, eye movement and gaze, among other things. The traditional eye control system adopts pupil monitoring, pupil positioning, Hough algorithm and other methods. This study will focus on a CNN-based eye control system. First of all, the authors present the CNN model, which is effective in image identification, target detection and tracking. Furthermore, the CNN-based eye control system is separated into three categories: semantic information, monocular/binocular and full-face. Finally, three challenges linked to the development of an eye control system based on a CNN are discussed, along with possible solutions.

This research can provide theoretical and engineering basis for the eye control system platform. In addition, it also summarizes the ideas of predecessors to support the development of future research.

The purpose of this paper is to verify the correctness and feasibility of simultaneous localization and mapping (SLAM) algorithm based on red-green-blue depth (RGB-D) camera in high precision navigation and localization of celestial exploration rover.

First, a positioning algorithm based on depth camera is proposed. Second, the realization method is described from the five aspects of feature detection method, feature point matching, point cloud mapping, motion estimation and high precision optimization. Feature detection: taking the precision, real-time and motion basics as the comprehensive consideration, the ORB (oriented FAST and rotated BRIEF) features extraction method is adopted; feature point matching: solves the similarity measure of the feature descriptor vector and how to remove the mismatch point; point cloud mapping: the two-dimensional information on RGB and the depth information on D corresponding; motion estimation: the iterative closest point algorithm is used to solve point set registration; and high precision optimization: optimized by using the graph optimization method.

The proposed high-precision SLAM algorithm is very effective for solving high precision navigation and positioning of celestial exploration rover.

In this paper, the simulation validation is based on an open source data set for testing; the physical verification is based on the existing unmanned vehicle platform to simulate the celestial exploration rover.

This paper presents a RGB-D camera-based navigation algorithm, which can be obtained by simulation experiment and physical verification. The real-time and accuracy of the algorithm are well behaved and have strong applicability, which can support the tests and experiments on hardware platform and have a better environmental adaptability.

The proposed SLAM algorithm can deal with the high precision navigation and positioning of celestial exploration rover effectively. Taking into account the current wide application prospect of computer vision, the method in this paper can provide a study foundation for the deep space probe.

Many popular simultaneous localization and mapping (SLAM) techniques have low accuracy, especially when localizing environments containing dynamically moving objects since their presence can potentially cause inaccurate data associations. To address this issue, the proposed FADM-SLAM system aims to improve the accuracy of SLAM techniques in environments containing dynamically moving objects. It uses a pipeline of feature-based approaches accompanied by sparse optical flow and multi-view geometry as constraints to achieve this goal.

FADM-SLAM, which works with monocular, stereo and RGB-D sensors, combines an instance segmentation network incorporating an intelligent motion detection strategy (iM) with an optical flow technique to improve location accuracy. The proposed AS-SLAM system comprises four principal modules, which are the optical flow mask and iM, the ego motion estimation, dynamic point detection and the feature-based extraction framework.

Experiment results using the publicly available RGBD-Bonn data set indicate that FADM-SLAM outperforms established visual SLAM systems in highly dynamic conditions.

In summary, the first module generates the indication of dynamic objects by using the optical flow and iM with geometric-wise segmentation, which is then used by the second module to compute the starting point of a posture. The third module, meanwhile, first searches for the dynamic feature points in the environment, and second, eliminates them from further processing. An algorithm based on epipolar constraints is implemented to do this. In this way, only the static feature points are retained, which are then fed to the fourth module for extracting important features.