Search results

The purpose of this paper is to summarize the existing point cloud target detection algorithms based on deep learning, and provide reference for researchers in related fields. In recent years, with its outstanding performance in target detection of 2D images, deep learning technology has been applied in light detection and ranging (LiDAR) point cloud data to improve the automation and intelligence level of target detection. However, there are still some difficulties and room for improvement in target detection from the 3D point cloud. In this paper, the vehicle LiDAR target detection method is chosen as the research subject.

Firstly, the challenges of applying deep learning to point cloud target detection are described; secondly, solutions in relevant research are combed in response to the above challenges. The currently popular target detection methods are classified, among which some are compared with illustrate advantages and disadvantages. Moreover, approaches to improve the accuracy of network target detection are introduced.

Finally, this paper also summarizes the shortcomings of existing methods and signals the prospective development trend.

This paper introduces some existing point cloud target detection methods based on deep learning, which can be applied to a driverless, digital map, traffic monitoring and other fields, and provides a reference for researchers in related fields.

This paper aims to present an object detection methodology to categorize 3D object models in an efficient manner. The authors propose a dynamically generated hierarchical architecture to compute very fast objects’ 3D pose for mobile service robots to grasp them.

The methodology used in this study is based on a dynamic pyramid search and fast template representation, metadata and context-free grammars. In the experiments, the authors use an omnidirectional KUKA mobile manipulator equipped with an RGBD camera, to localize objects requested by humans. The proposed architecture is based on efficient object detection and visual servoing. In the experiments, the robot successfully finds 3D poses. The present proposal is not restricted to specific robots or objects and can grow as much as needed.

The authors present the dynamic categorization using context-free grammars and 3D object detection, and through several experiments, the authors perform a proof of concept. The authors obtained promising results, showing that their methods can scale to more complex scenes and they can be used in future applications in real-world scenarios where mobile robot are needed in areas such as service robots or industry in general.

The experiments were carried out using a mobile KUKA youBot. Scalability and more robust algorithms will improve the present proposal. In the first stage, the authors carried out an experimental validation.

The current proposal describes a scalable architecture, where more agents can be added or reprogrammed to handle more complicated tasks.

The main contribution of this study resides in the dynamic categorization scheme for fast detection of 3D objects, and the issues and experiments carried out to test the viability of the methods. Usually, state-of-the-art treats categories as rigid and make static queries to datasets. In the present approach, there are no fixed categories and they are created and combined on the fly to speed up detection.

The purpose of this paper is to develop a robot vision system for surface defect detection of 3D objects. It aims at the ill‐defined qualitative items such as stains and scratches.

A robot vision system for surface defect detection may counter: high surface reflection at some viewing angles; and no reference markers in any sensed images for matching. A filtering process is used to separate the illumination and reflection components of an image. An automatic marker‐selection process and a template‐matching method are then proposed for image registration and anomaly detection in reflection‐free images.

Tests were performed on a variety of hand‐held electronic devices such as cellular phones. Experimental results show that the proposed system can reliably avoid reflection surfaces and effectively identify small local defects on the surfaces in different viewing angles.

The results have practical implications for industrial objects with arbitrary surfaces.

Traditional visual inspection systems mainly work for two‐dimensional planar surfaces such as printed circuit boards and wafers. The proposed system can find the viewing angles with minimum surface reflection and detect small local defects under image misalignment for three‐dimensional objects.

Detecting objects in images and videos is a difficult task that has challenged the field of computer vision. Most of the algorithms for object detection are sensitive to background clutter and occlusion, and cannot localize the edge of the object. An object's shape is typically the most discriminative cue for its recognition by humans. The purpose of this paper is to introduce a model‐based object detection method which uses only shape‐fragment features.

The object shape model is learned from a small set of training images and all object models are composed of shape fragments. The model of the object is in multi‐scales.

The major contributions of this paper are the application of learned shape fragments‐based model for object detection in complex environment and a novel two‐stage object detection framework.

The results presented in this paper are competitive with other state‐of‐the‐art object detection methods.

The purpose of this paper is to explore the role that computer vision can play within new industrial paradigms such as Industry 4.0 and in particular to support production line improvements to achieve flexible manufacturing. As Industry 4.0 requires “big data”, it is accepted that computer vision could be one of the tools for its capture and efficient analysis. RGB-D data gathered from real-time machine vision systems such as Kinect ® can be processed using computer vision techniques.

This research exploits RGB-D cameras such as Kinect® to investigate the feasibility of using computer vision techniques to track the progress of a manual assembly task on a production line. Several techniques to track the progress of a manual assembly task are presented. The use of CAD model files to track the manufacturing tasks is also outlined.

This research has found that RGB-D cameras can be suitable for object recognition within an industrial environment if a number of constraints are considered or different devices/techniques combined. Furthermore, through the use of a HMM inspired state-based workflow, the algorithm presented in this paper is computationally tractable.

Processing of data from robust and cheap real-time machine vision systems could bring increased understanding of production line features. In addition, new techniques that enable the progress tracking of manual assembly sequences may be defined through the further analysis of such visual data. The approaches explored within this paper make a contribution to the utilisation of visual information “big data” sets for more efficient and automated production.

This paper aims to design a deep neural network for object instance segmentation and six-dimensional (6D) pose estimation in cluttered scenes and apply the proposed method in real-world robotic autonomous grasping of household objects.

A novel deep learning method is proposed for instance segmentation and 6D pose estimation in cluttered scenes. An iterative pose refinement network is integrated with the main network to obtain more robust final pose estimation results for robotic applications. To train the network, a technique is presented to generate abundant annotated synthetic data consisting of RGB-D images and object masks in a fast manner without any hand-labeling. For robotic grasping, the offline grasp planning based on eigengrasp planner is performed and combined with the online object pose estimation.

The experiments on the standard pose benchmarking data sets showed that the method achieves better pose estimation and time efficiency performance than state-of-art methods with depth-based ICP refinement. The proposed method is also evaluated on a seven DOFs Kinova Jaco robot with an Intel Realsense RGB-D camera, the grasping results illustrated that the method is accurate and robust enough for real-world robotic applications.

A novel 6D pose estimation network based on the instance segmentation framework is proposed and a neural work-based iterative pose refinement module is integrated into the method. The proposed method exhibits satisfactory pose estimation and time efficiency for the robotic grasping.

Wheeled mobile robots (WMR) are the most widely used robots. Avoiding obstacles in unstructured environments, especially dynamic obstacles such as pedestrians, is a serious challenge for WMR. This paper aims to present a hybrid obstacle avoidance method that combines an informed-rapidly exploring random tree* algorithm with a three-dimensional (3D)-object detection approach and model prediction controller (MPC) to conduct obstacle perception, collision-free path planning and obstacle avoidance for WMR in unstructured environments.

Given a reference orientation and speed, the hybrid method uses parametric ellipses to represent obstacle expansion boundaries based on the 3D target detection results, and a collision-free reference path is planned. Then, the authors build on a model predictive control for tracking the collision-free reference path by incorporating the distance between the robot and obstacles. The proposed framework is a mapless method for WMR.

The authors present experimental results with a mobile robot for obstacle avoidance in indoor environments crowded with obstacles, such as chairs and pedestrians. The results show that the proposed hybrid obstacle avoidance method can satisfy the application requirements of mobile robots in unstructured environments.

In this study, the parameter ellipse is used to represent the area occupied by the obstacle, which takes the velocity as the parameter. Therefore, the motion direction and position of dynamic obstacles can be considered in the planning stage, which enhances the success rate of obstacle avoidance. In addition, the distance between the obstacle and robot is increased in the MPC optimization function to ensure a safe distance between the robot and the obstacle.

The paper aims to address the tracking algorithm based on deep learning and four deep learning tracking models developed. They compared with each other to prevent collision and to obtain target tracking in autonomous aircraft.

First, to follow the visual target, the detection methods were used and then the tracking methods were examined. Here, four models (deep convolutional neural networks (DCNN), deep convolutional neural networks with fine-tuning (DCNNFN), transfer learning with deep convolutional neural network (TLDCNN) and fine-tuning deep convolutional neural network with transfer learning (FNDCNNTL)) were developed.

The training time of DCNN took 9 min 33 s, while the accuracy percentage was calculated as 84%. In DCNNFN, the training time of the network was calculated as 4 min 26 s and the accuracy percentage was 91%. The training of TLDCNN) took 34 min and 49 s and the accuracy percentage was calculated as 95%. With FNDCNNTL, the training time of the network was calculated as 34 min 33 s and the accuracy percentage was nearly 100%.

Compared to the results in the literature ranging from 89.4% to 95.6%, using FNDCNNTL, better results were found in the paper.

Picking up pistons in arbitrary poses is an important step on car engine assembly line. The authors usually use vision system to estimate the pose of the pistons and then guide a stable grasp. However, a piston in some poses, e.g. the mouth of the piston faces forward, is hardly to be directly grasped by the gripper. Thus, we need to reorient the piston to achieve a desired pose, i.e. let its mouth face upward, for grasping.

This paper aims to present a vision-based picking system that can grasp pistons in arbitrary poses. The whole picking process is divided into two stages. At localization stage, a hierarchical approach is proposed to estimate the piston’s pose from image which usually involves both heavy noise and edge distortions. At grasping stage, multi-step robotic manipulations are designed to enable the piston to follow a nominal trajectory to reach to the minimum of the distance between the piston’s center and the support plane. That is, under the design input, the piston would be pushed to achieve a desired orientation.

A target piston in arbitrary poses would be picked from the conveyor belt by the gripper with the proposed method.

The designed robotic bin-picking system using vision is an advantage in terms of flexibility in automobile manufacturing industry.

The authors develop a methodology that uses a pneumatic gripper and 2D vision information for picking up multiple pistons in arbitrary poses. The rough pose of the parts are detected based on a hierarchical approach for detection of multiple ellipses in the environment that usually involve edge distortions. The pose uncertainties of the piston are eliminated by multi-step robotic manipulations.

Template matching is one of the most suitable choices for full six degrees of freedom pose estimation in many practical industrial applications. However, the increasing number of templates while dealing with a wide range of viewpoint changes results in a long runtime, which may not meet the real-time requirements. This paper aims to improve matching efficiency while maintaining sample resolution and matching accuracy.

A multi-pyramid-based hierarchical template matching strategy is proposed. Three pyramids are established at the sphere subdivision, radius and in-plane rotation levels during the offline template render stage. Then, a hierarchical template matching is performed from the highest to the lowest level in each pyramid, narrowing the global search space and expanding the local search space. The initial search parameters at the top level can be determined by the preprocessing of the YOLOv3 object detection network to further improve real-time performance.

Experimental results show that this matching strategy takes only 100 ms under 100k templates without loss of accuracy, promising for real industrial applications. The authors further validated the approach by applying it to a real robot grasping task.

The matching framework in this paper improves the template matching efficiency by two orders of magnitude and is validated using a common template definition and viewpoint sampling methods. In addition, it can be easily adapted to other template definitions and viewpoint sampling methods.