Search results

Visual feedback control is a promising solution for robots work in unstructured environments, and this is accomplished by estimation of the time derivative relationship between the image features and the robot moving. While some of the drawbacks associated with most visual servoing (VS) approaches include the vision–motor mapping computation and the robots’ dynamic performance, the problem of designing optimal and more effective VS systems still remains challenging. Thus, the purpose of this paper is to propose and evaluate the VS method for robots in an unstructured environment.

This paper presents a new model-free VS control of a robotic manipulator, for which an adaptive estimator aid by network learning is proposed using online estimation of the vision–motor mapping relationship in an environment without the knowledge of statistical noise. Based on the adaptive estimator, a model-free VS schema was constructed by introducing an active disturbance rejection control (ADRC). In our schema, the VS system was designed independently of the robot kinematic model.

The various simulations and experiments were conducted to verify the proposed approach by using an eye-in-hand robot manipulator without calibration and vision depth information, which can improve the autonomous maneuverability of the robot and also allow the robot to adapt its motion according to the image feature changes in real time. In the current method, the image feature trajectory was stable in the camera field range, and the robot’s end motion trajectory did not exhibit shock retreat. The results showed that the steady-state errors of image features was within 19.74 pixels, the robot positioning was stable within 1.53 mm and 0.0373 rad and the convergence rate of the control system was less than 7.21 s in real grasping tasks.

Compared with traditional Kalman filtering for image-based VS and position-based VS methods, this paper adopts the model-free VS method based on the adaptive mapping estimator combination with the ADRC controller, which is effective for improving the dynamic performance of robot systems. The proposed model-free VS schema is suitable for robots’ grasping manipulation in unstructured environments.

The swivel construction method is a specially designed process used to build bridges that cross rivers, valleys, railroads and other obstacles. To carry out this construction method safely, real-time monitoring of the bridge rotation process is required to ensure a smooth swivel operation without collisions. However, the traditional means of monitoring using Electronic Total Station tools cannot realize real-time monitoring, and monitoring using motion sensors or GPS is cumbersome to use.

This study proposes a monitoring method based on a series of computer vision (CV) technologies, which can monitor the rotation angle, velocity and inclination angle of the swivel construction in real-time. First, three proposed CV algorithms was developed in a laboratory environment. The experimental tests were carried out on a bridge scale model to select the outperformed algorithms for rotation, velocity and inclination monitor, respectively, as the final monitoring method in proposed method. Then, the selected method was implemented to monitor an actual bridge during its swivel construction to verify the applicability.

In the laboratory study, the monitoring data measured with the selected monitoring algorithms was compared with those measured by an Electronic Total Station and the errors in terms of rotation angle, velocity and inclination angle, were 0.040%, 0.040%, and −0.454%, respectively, thus validating the accuracy of the proposed method. In the pilot actual application, the method was shown to be feasible in a real construction application.

In a well-controlled laboratory the optimal algorithms for bridge swivel construction are identified and in an actual project the proposed method is verified. The proposed CV method is complementary to the use of Electronic Total Station tools, motion sensors, and GPS for safety monitoring of swivel construction of bridges. It also contributes to being a possible approach without data-driven model training. Its principal advantages are that it both provides real-time monitoring and is easy to deploy in real construction applications.

Construction progress monitoring (CPM) is considered a difficult and tedious task in construction projects, which focuses on identifying discrepancies between the as-built product and the as-planned design. Computer vision (CV) technology is applied to automate the CPM process. However, the synergy between the CV and CPM in literature and industry practice is lacking. This study aims to fulfil this research gap.

A Delphi qualitative approach was used in this study by conducting two interview rounds. The collected data was analysed using manual content analysis.

This study identified seven stages of CPM; data acquisition, information retrieval, verification, progress estimation and comparison, visualisation of the results and schedule updating. Factors such as higher accuracy in data, less labourious process, efficiency and near real-time access are some of the significant enablers in instigating CV for CPM. Major challenges identified were occlusions and lighting issues in the site images and lack of support from the management. The challenges can be easily overcome by implementing suitable strategies such as familiarisation of the workforce with CV technology and application of CV research for the construction industry to grow with the technology in line with other industries.

This study addresses the gap pertaining to the synergy between the CV in CPM literature and the industry practice. This research contributes by enabling the construction personnel to identify the shortcomings and the opportunities to apply automated technologies concerning each stage in the progress monitoring process.

Images taken from construction site interiors often suffer from low illumination and poor natural colors, which restrict their application for high-level site management purposes. The state-of-the-art low-light image enhancement method provides promising image enhancement results. However, they generally require a longer execution time to complete the enhancement. This study aims to develop a refined image enhancement approach to improve execution efficiency and performance accuracy.

To develop the refined illumination enhancement algorithm named enhanced illumination quality (EIQ), a quadratic expression was first added to the initial illumination map. Subsequently, an adjusted weight matrix was added to improve the smoothness of the illumination map. A coordinated descent optimization algorithm was then applied to minimize the processing time. Gamma correction was also applied to further enhance the illumination map. Finally, a frame comparing and averaging method was used to identify interior site progress.

The proposed refined approach took around 4.36–4.52 s to achieve the expected results while outperforming the current low-light image enhancement method. EIQ demonstrated a lower lightness-order error and provided higher object resolution in enhanced images. EIQ also has a higher structural similarity index and peak-signal-to-noise ratio, which indicated better image reconstruction performance.

The proposed approach provides an alternative to shorten the execution time, improve equalization of the illumination map and provide a better image reconstruction. The approach could be applied to low-light video enhancement tasks and other dark or poor jobsite images for object detection processes.

The purpose of this study is to establish a method for accurately extracting torch and seam features. This will improve the quality of narrow gap welding. An adaptive deflection correction system based on passive light vision sensors was designed using the Halcon software from MVtec Germany as a platform.

This paper proposes an adaptive correction system for welding guns and seams divided into image calibration and feature extraction. In the image calibration method, the field of view distortion because of the position of the camera is resolved using image calibration techniques. In the feature extraction method, clear features of the weld gun and weld seam are accurately extracted after processing using algorithms such as impact filtering, subpixel (XLD), Gaussian Laplacian and sense region for the weld gun and weld seam. The gun and weld seam centers are accurately fitted using least squares. After calculating the deviation values, the error values are monitored, and error correction is achieved by programmable logic controller (PLC) control. Finally, experimental verification and analysis of the tracking errors are carried out.

The results show that the system achieves great results in dealing with camera aberrations. Weld gun features can be effectively and accurately identified. The difference between a scratch and a weld is effectively distinguished. The system accurately detects the center features of the torch and weld and controls the correction error to within 0.3mm.

An adaptive correction system based on a passive light vision sensor is designed which corrects the field-of-view distortion caused by the camera’s position deviation. Differences in features between scratches and welds are distinguished, and image features are effectively extracted. The final system weld error is controlled to 0.3 mm.

This study aims to investigate the effect of speckle pattern on displacement measurements using different speckle diameters and coverage ratios.

In order to compare the coverage ratio and speckle diameter during the evaluation of the correlation of digital images (DIC) study, template speckle plates were produced on a computer numerical control (CNC) punch press with 600 punches per minute. After the speckle plates were manufactured, the speckled pattern was randomly painted on a plain white side through the manufactured template plates, and then tensile tests were performed under the same loading conditions for each sample to observe displacement variation via correlation parameters.

During the manufacturing of templates with thin plates, a punch diameter of less than 1.7 mm will cause tool failure; therefore, uniform speckle size can be assessed before operation. A higher coverage ratio resulted in more accurate and reliable results in displacement data. With smaller coverage, the facet size should be increased to achieve favorable results.

If thick template plates are selected, speckle painting cannot be done properly; therefore, template thickness shall also be assessed before operation.

For randomly distributed DIC templates, increasing coverage beyond 50% does not make sense due to difficulties in the production process in the punch press.

Evaluating DIC results via templates manufactured in a punch press with different speckle diameters and coverage ratios is a new topic in literature.

Because of the advantages of high deposition efficiency and low manufacturing cost compared with other additive technologies, robotic wire arc additive manufacturing (WAAM) technology has been widely applied for fabricating medium- to large-scale metallic components. The additive manufacturing (AM) method is a relatively complex process, which involves the workpiece modeling, conversion of the model file, slicing, path planning and so on. Then the structure is formed by the accumulated weld bead. However, the poor forming accuracy of WAAM usually leads to severe dimensional deviation between the as-built and the predesigned structures. This paper aims to propose a visual sensing technology and deep learning–assisted WAAM method for fabricating metallic structure, to simplify the complex WAAM process and improve the forming accuracy.

Instead of slicing of the workpiece modeling and generating all the welding torch paths in advance of the fabricating process, this method is carried out by adding the feature point regression branch into the Yolov5 algorithm, to detect the feature point from the images of the as-built structure. The coordinates of the feature points of each deposition layer can be calculated automatically. Then the welding torch trajectory for the next deposition layer is generated based on the position of feature point.

The mean average precision score of modified YOLOv5 detector is 99.5%. Two types of overhanging structures have been fabricated by the proposed method. The center contour error between the actual and theoretical is 0.56 and 0.27 mm in width direction, and 0.43 and 0.23 mm in height direction, respectively.

The fabrication of circular overhanging structures without using the complicate slicing strategy, turning table or other extra support verified the possibility of the robotic WAAM system with deep learning technology.

In the context of fixed-wing aircraft wing assembly, there is a need for a rapid and precise measurement technique to determine the center distance between two double-hole components. This paper aims to propose an optical-based spatial point distance measurement technique using the spatial triangulation method. The purpose of this paper is to design a specialized measurement system, specifically a spherically mounted retroreflector nest (SMR nest), equipped with two laser displacement sensors and a rotary encoder as the core to achieve accurate distance measurements between the double holes.

To develop an efficient and accurate measurement system, the paper uses a combination of laser displacement sensors and a rotary encoder within the SMR nest. The system is designed, implemented and tested to meet the requirements of precise distance measurement. Software and hardware components have been developed and integrated for validation.

The optical-based distance measurement system achieves high precision at 0.04 mm and repeatability at 0.02 mm within a range of 412.084 mm to 1,590.591 mm. These results validate its suitability for efficient assembly processes, eliminating repetitive errors in aircraft wing assembly.

This paper proposes an optical-based spatial point distance measurement technique, as well as a unique design of a SMR nest and the introduction of two novel calibration techniques, all of which are validated by the developed software and hardware platform.

Safety training can effectively facilitate workers’ safety awareness and prevent injuries and fatalities on construction sites. Traditional training methods are time-consuming, low participation, and less interaction, which is not suitable for students who are born in Generation Z (Gen Z) and expect to be positively engaged in the learning process. With the characteristic of immersive, interaction, and imagination, virtual reality (VR) has become a promising training method. The purpose of this study is to explore Gen Z students’ learning differences under VR and traditional conditions and determine whether VR technology is more suitable for Gen Z students.

This paper designed a comparison experiment that includes three training conditions: VR-based, classroom lecturing, and on-site practice. 32 sophomore students were divided into four groups and received different training methods. The eye movement data and hazard-identification index (HII) scores from four groups were collected to measure their hazard-identification ability. The differences between the participants before and after the test were tested by paired sample t-test, and the differences between the groups after the test were analyzed by one-way Welch’s analysis of variance (ANOVA) test.

The statistical findings showed that participants under VR technology condition spent less time finding and arriving at the Areas of Interest (AOIs). Both the eye movement data and HII scores indicated that VR-based safety training is an alternative approach for Gen Z students to traditional safety training methods.

These findings contribute to the theoretical implications by proving the applicability of VR technology to Gen Z students and empirical implications by guiding colleges and universities to design attractive safety training lessons.

The fourth industrial revolution and digital transformation have caused paradigm changes in the procedures of goods production and services through disruptive technologies, and they have formed new methods for business models. Health and medicine fields have been under the effect of these technology advancements. The concept of smart hospital is formed according to these technological transformations. The aim of this research, other than explanation of smart hospital components, is to present a model for evaluating a hospital readiness for becoming a smart hospital.

This research is an applied one, and has been carried out in three phases and according to design science research. Based on the previous studies, in the first phase, the components and technologies effecting a smart hospital are recognized. In the second phase, the extracted components are prioritized using type-2 fuzzy analytic hierarchical process based on the opinion of experts; later, the readiness model is designed. In the third phase, the presented model would be tested in a hospital.

The research results showed that the technologies of internet of things, robotics, artificial intelligence, radio-frequency identification as well as augmented and virtual reality had the most prominence in a smart hospital.

The innovation and originality of the forthcoming research is to explain the concept of smart hospital, to rank its components and to provide a model for evaluating the readiness of smart hospital. Contribution of this research in terms of theory explains the concept of smart hospital and in terms of application presents a model for assessing the readiness of smart hospitals.