Search results

Vision, audition, olfactory, tactile and taste are five important senses that human uses to interact with the real world. As facing more and more complex environments, a sensing system is essential for intelligent robots with various types of sensors. To mimic human-like abilities, sensors similar to human perception capabilities are indispensable. However, most research only concentrated on analyzing literature on single-modal sensors and their robotics application.

This study presents a systematic review of five bioinspired senses, especially considering a brief introduction of multimodal sensing applications and predicting current trends and future directions of this field, which may have continuous enlightenments.

This review shows that bioinspired sensors can enable robots to better understand the environment, and multiple sensor combinations can support the robot’s ability to behave intelligently.

The review starts with a brief survey of the biological sensing mechanisms of the five senses, which are followed by their bioinspired electronic counterparts. Their applications in the robots are then reviewed as another emphasis, covering the main application scopes of localization and navigation, objection identification, dexterous manipulation, compliant interaction and so on. Finally, the trends, difficulties and challenges of this research were discussed to help guide future research on intelligent robot sensors.

This paper presents a survey of research into interactive robotic systems for the purpose of identifying the state of the art capabilities as well as the extant gaps in this emerging field. Communication is multimodal. Multimodality is a representation of many modes chosen from rhetorical aspects for its communication potentials. The author seeks to define the available automation capabilities in communication using multimodalities that will support a proposed Interactive Robot System (IRS) as an AI mounted robotic platform to advance the speed and quality of military operational and tactical decision making.

This review will begin by presenting key developments in the robotic interaction field with the objective of identifying essential technological developments that set conditions for robotic platforms to function autonomously. After surveying the key aspects in Human Robot Interaction (HRI), Unmanned Autonomous System (UAS), visualization, Virtual Environment (VE) and prediction, the paper then proceeds to describe the gaps in the application areas that will require extension and integration to enable the prototyping of the IRS. A brief examination of other work in HRI-related fields concludes with a recapitulation of the IRS challenge that will set conditions for future success.

Using insights from a balanced cross section of sources from the government, academic, and commercial entities that contribute to HRI a multimodal IRS in military communication is introduced. Multimodal IRS (MIRS) in military communication has yet to be deployed.

Multimodal robotic interface for the MIRS is an interdisciplinary endeavour. This is not realistic that one can comprehend all expert and related knowledge and skills to design and develop such multimodal interactive robotic interface. In this brief preliminary survey, the author has discussed extant AI, robotics, NLP, CV, VDM, and VE applications that is directly related to multimodal interaction. Each mode of this multimodal communication is an active research area. Multimodal human/military robot communication is the ultimate goal of this research.

A multimodal autonomous robot in military communication using speech, images, gestures, VST and VE has yet to be deployed. Autonomous multimodal communication is expected to open wider possibilities for all armed forces. Given the density of the land domain, the army is in a position to exploit the opportunities for human–machine teaming (HMT) exposure. Naval and air forces will adopt platform specific suites for specially selected operators to integrate with and leverage this emerging technology. The possession of a flexible communications means that readily adapts to virtual training will enhance planning and mission rehearsals tremendously.

Interaction, perception, cognition and visualization based multimodal communication system is yet missing. Options to communicate, express and convey information in HMT setting with multiple options, suggestions and recommendations will certainly enhance military communication, strength, engagement, security, cognition, perception as well as the ability to act confidently for a successful mission.

The objective is to develop a multimodal autonomous interactive robot for military communications. This survey reports the state of the art, what exists and what is missing, what can be done and possibilities of extension that support the military in maintaining effective communication using multimodalities. There are some separate ongoing progresses, such as in machine-enabled speech, image recognition, tracking, visualizations for situational awareness, and virtual environments. At this time, there is no integrated approach for multimodal human robot interaction that proposes a flexible and agile communication. The report briefly introduces the research proposal about multimodal interactive robot in military communication.

Crystallization is the process widely used for components separation and solids purification. The systems for crystallization process evaluation applied so far, involve numerous non-invasive tomographic measurement techniques which suffers from some reported problems. The purpose of this paper is to show the abilities of three-dimensional Electrical Capacitance Tomography (3D ECT) in the context of non-invasive and non-intrusive visualization of crystallization processes. Multiple aspects and problems of ECT imaging, as well as the computer model design to work with the high relative permittivity liquids, have been pointed out.

To design the most efficient (from a mechanical and electrical point of view) 3D ECT sensor structure, the high-precise impedance meter was applied. The three types of sensor were designed, built, and tested. To meet the new concept requirements, the dedicated ECT device has been constructed.

It has been shown that the ECT technique can be applied to the diagnosis of crystallization. The crystals distribution can be identified using this technique. The achieved measurement resolution allows detecting the localization of crystals. The usage of stabilized electrodes improves the sensitivity of the sensor and provides the images better suitable for further analysis.

The dedicated 3D ECT sensor construction has been proposed to increase its sensitivity in the border area, where the crystals grow. Regarding this feature, some new algorithms for the potential field distribution and the sensitivity matrix calculation have been developed. The adaptation of the iterative 3D image reconstruction process has also been described.

This pragmatic research paper aims to unravel the smart vision-based method (SVBM), an AI program to correlate the computer vision (recorded and live videos using mobile and embedded cameras) that aids in manual lifting human pose deduction, analysis and training in the construction sector.

Using a pragmatic approach combined with the literature review, this study discusses the SVBM. The research method includes a literature review followed by a pragmatic approach and lab validation of the acquired data. Adopting the practical approach, the authors of this article developed an SVBM, an AI program to correlate computer vision (recorded and live videos using mobile and embedded cameras).

Results show that SVBM observes the relevant events without additional attachments to the human body and compares them with the standard axis to identify abnormal postures using mobile and other cameras. Angles of critical nodal points are projected through human pose detection and calculating body part movement angles using a novel software program and mobile application. The SVBM demonstrates its ability to data capture and analysis in real-time and offline using videos recorded earlier and is validated for program coding and results repeatability.

Literature review methodology limitations include not keeping in phase with the most updated field knowledge. This limitation is offset by choosing the range for literature review within the last two decades. This literature review may not have captured all published articles because the restriction of database access and search was based only on English. Also, the authors may have omitted fruitful articles hiding in a less popular journal. These limitations are acknowledged. The critical limitation is that the trust, privacy and psychological issues are not addressed in SVBM, which is recognised. However, the benefits of SVBM naturally offset this limitation to being adopted practically.

The theoretical and practical implications include customised and individualistic prediction and preventing most posture-related hazardous behaviours before a critical injury happens. The theoretical implications include mimicking the human pose and lab-based analysis without attaching sensors that naturally alter the working poses. SVBM would help researchers develop more accurate data and theoretical models close to actuals.

By using SVBM, the possibility of early deduction and prevention of musculoskeletal disorders is high; the social implications include the benefits of being a healthier society and health concerned construction sector.

Human pose detection, especially joint angle calculation in a work environment, is crucial to early deduction of muscoloskeletal disorders. Conventional digital technology-based methods to detect pose flaws focus on location information from wearables and laboratory-controlled motion sensors. For the first time, this paper presents novel computer vision (recorded and live videos using mobile and embedded cameras) and digital image-related deep learning methods without attachment to the human body for manual handling pose deduction and analysis of angles, neckline and torso line in an actual construction work environment.

The future construction site will be pervasive, context aware and embedded with intelligence. The purpose of this paper is to explore and define the concept of the digital skin of the future smart construction site.

The paper provides a systematic and hierarchical classification of 114 articles from both industry and academia on the digital skin concept and evaluates them. The hierarchical classification is based on application areas relevant to construction, such as augmented reality, building information model-based visualisation, labour tracking, supply chain tracking, safety management, mobile equipment tracking and schedule and progress monitoring. Evaluations of the research papers were conducted based on three pillars: validation of technological feasibility, onsite application and user acceptance testing.

Technologies learned about in the literature review enabled the envisaging of the pervasive construction site of the future. The paper presents scenarios for the future context-aware construction site, including the construction worker, construction procurement management and future real-time safety management systems.

Based on the gaps identified by the review in the body of knowledge and on a broader analysis of technology diffusion, the paper highlights the research challenges to be overcome in the advent of digital skin. The paper recommends that researchers follow a coherent process for smart technology design, development and implementation in order to achieve this vision for the construction industry.

Intelligent and connected vehicle technology is in the ascendant. High-level autonomous driving places more stringent requirements on the accuracy and reliability of environmental perception. Existing research works on multitarget tracking based on multisensor fusion mostly focuses on the vehicle perspective, but limited by the principal defects of the vehicle sensor platform, it is difficult to comprehensively and accurately describe the surrounding environment information.

In this paper, a multitarget tracking method based on roadside multisensor fusion is proposed, including a multisensor fusion method based on measurement noise adaptive Kalman filtering, a global nearest neighbor data association method based on adaptive tracking gate, and a Track life cycle management method based on M/N logic rules.

Compared with fixed-size tracking gates, the adaptive tracking gates proposed in this paper can comprehensively improve the data association performance in the multitarget tracking process. Compared with single sensor measurement, the proposed method improves the position estimation accuracy by 13.5% and the velocity estimation accuracy by 22.2%. Compared with the control method, the proposed method improves the position estimation accuracy by 23.8% and the velocity estimation accuracy by 8.9%.

A multisensor fusion method with adaptive Kalman filtering of measurement noise is proposed to realize the adaptive adjustment of measurement noise. A global nearest neighbor data association method based on adaptive tracking gate is proposed to realize the adaptive adjustment of the tracking gate.

The purpose of this paper is to develop a proof-of-concept (POC) Forward Collision Warning (FWC) system for the motorcyclist, which determines a potential clash based on time-to-collision and trajectory of both the detected and ego vehicle (motorcycle).

This comes in three approaches. First, time-to-collision value is to be calculated based on low-cost camera video input. Second, the trajectory of the detected vehicle is predicted based on video data in the 2 D pixel coordinate. Third, the trajectory of the ego vehicle is predicted via the lean direction of the motorcycle from a low-cost inertial measurement unit sensor.

This encompasses a comprehensive Advanced FWC system which is an amalgamation of the three approaches mentioned above. First, to predict time-to-collision, nested Kalman filter and vehicle detection is used to convert image pixel matrix to relative distance, velocity and time-to-collision data. Next, for trajectory prediction of detected vehicles, a few algorithms were compared, and it was found that long short-term memory performs the best on the data set. The last finding is that to determine the leaning direction of the ego vehicle, it is better to use lean angle measurement compared to riding pattern classification.

The value of this paper is that it provides a POC FWC system that considers time-to-collision and trajectory of both detected and ego vehicle (motorcycle).