Search results

This paper presents a survey of research into interactive robotic systems for the purpose of identifying the state of the art capabilities as well as the extant gaps in this emerging field. Communication is multimodal. Multimodality is a representation of many modes chosen from rhetorical aspects for its communication potentials. The author seeks to define the available automation capabilities in communication using multimodalities that will support a proposed Interactive Robot System (IRS) as an AI mounted robotic platform to advance the speed and quality of military operational and tactical decision making.

This review will begin by presenting key developments in the robotic interaction field with the objective of identifying essential technological developments that set conditions for robotic platforms to function autonomously. After surveying the key aspects in Human Robot Interaction (HRI), Unmanned Autonomous System (UAS), visualization, Virtual Environment (VE) and prediction, the paper then proceeds to describe the gaps in the application areas that will require extension and integration to enable the prototyping of the IRS. A brief examination of other work in HRI-related fields concludes with a recapitulation of the IRS challenge that will set conditions for future success.

Using insights from a balanced cross section of sources from the government, academic, and commercial entities that contribute to HRI a multimodal IRS in military communication is introduced. Multimodal IRS (MIRS) in military communication has yet to be deployed.

Multimodal robotic interface for the MIRS is an interdisciplinary endeavour. This is not realistic that one can comprehend all expert and related knowledge and skills to design and develop such multimodal interactive robotic interface. In this brief preliminary survey, the author has discussed extant AI, robotics, NLP, CV, VDM, and VE applications that is directly related to multimodal interaction. Each mode of this multimodal communication is an active research area. Multimodal human/military robot communication is the ultimate goal of this research.

A multimodal autonomous robot in military communication using speech, images, gestures, VST and VE has yet to be deployed. Autonomous multimodal communication is expected to open wider possibilities for all armed forces. Given the density of the land domain, the army is in a position to exploit the opportunities for human–machine teaming (HMT) exposure. Naval and air forces will adopt platform specific suites for specially selected operators to integrate with and leverage this emerging technology. The possession of a flexible communications means that readily adapts to virtual training will enhance planning and mission rehearsals tremendously.

Interaction, perception, cognition and visualization based multimodal communication system is yet missing. Options to communicate, express and convey information in HMT setting with multiple options, suggestions and recommendations will certainly enhance military communication, strength, engagement, security, cognition, perception as well as the ability to act confidently for a successful mission.

The objective is to develop a multimodal autonomous interactive robot for military communications. This survey reports the state of the art, what exists and what is missing, what can be done and possibilities of extension that support the military in maintaining effective communication using multimodalities. There are some separate ongoing progresses, such as in machine-enabled speech, image recognition, tracking, visualizations for situational awareness, and virtual environments. At this time, there is no integrated approach for multimodal human robot interaction that proposes a flexible and agile communication. The report briefly introduces the research proposal about multimodal interactive robot in military communication.

The successful and commercial use of self-driving/driverless/unmanned/automated car will make human life easier. The paper aims to discuss this issue.

This paper reviews the key technology of a self-driving car. In this paper, the four key technologies in self-driving car, namely, car navigation system, path planning, environment perception and car control, are addressed and surveyed. The main research institutions and groups in different countries are summarized. Finally, the debates of self-driving car are discussed and the development trend of self-driving car is predicted.

This paper analyzes the key technology of self-driving car and illuminates the state-of-art of the self-driving car.

The main research contents and key technology have been introduced. The research progress as well as the research institution has been summarized.

The future construction site will be pervasive, context aware and embedded with intelligence. The purpose of this paper is to explore and define the concept of the digital skin of the future smart construction site.

The paper provides a systematic and hierarchical classification of 114 articles from both industry and academia on the digital skin concept and evaluates them. The hierarchical classification is based on application areas relevant to construction, such as augmented reality, building information model-based visualisation, labour tracking, supply chain tracking, safety management, mobile equipment tracking and schedule and progress monitoring. Evaluations of the research papers were conducted based on three pillars: validation of technological feasibility, onsite application and user acceptance testing.

Technologies learned about in the literature review enabled the envisaging of the pervasive construction site of the future. The paper presents scenarios for the future context-aware construction site, including the construction worker, construction procurement management and future real-time safety management systems.

Based on the gaps identified by the review in the body of knowledge and on a broader analysis of technology diffusion, the paper highlights the research challenges to be overcome in the advent of digital skin. The paper recommends that researchers follow a coherent process for smart technology design, development and implementation in order to achieve this vision for the construction industry.

This pragmatic research paper aims to unravel the smart vision-based method (SVBM), an AI program to correlate the computer vision (recorded and live videos using mobile and embedded cameras) that aids in manual lifting human pose deduction, analysis and training in the construction sector.

Using a pragmatic approach combined with the literature review, this study discusses the SVBM. The research method includes a literature review followed by a pragmatic approach and lab validation of the acquired data. Adopting the practical approach, the authors of this article developed an SVBM, an AI program to correlate computer vision (recorded and live videos using mobile and embedded cameras).

Results show that SVBM observes the relevant events without additional attachments to the human body and compares them with the standard axis to identify abnormal postures using mobile and other cameras. Angles of critical nodal points are projected through human pose detection and calculating body part movement angles using a novel software program and mobile application. The SVBM demonstrates its ability to data capture and analysis in real-time and offline using videos recorded earlier and is validated for program coding and results repeatability.

Literature review methodology limitations include not keeping in phase with the most updated field knowledge. This limitation is offset by choosing the range for literature review within the last two decades. This literature review may not have captured all published articles because the restriction of database access and search was based only on English. Also, the authors may have omitted fruitful articles hiding in a less popular journal. These limitations are acknowledged. The critical limitation is that the trust, privacy and psychological issues are not addressed in SVBM, which is recognised. However, the benefits of SVBM naturally offset this limitation to being adopted practically.

The theoretical and practical implications include customised and individualistic prediction and preventing most posture-related hazardous behaviours before a critical injury happens. The theoretical implications include mimicking the human pose and lab-based analysis without attaching sensors that naturally alter the working poses. SVBM would help researchers develop more accurate data and theoretical models close to actuals.

By using SVBM, the possibility of early deduction and prevention of musculoskeletal disorders is high; the social implications include the benefits of being a healthier society and health concerned construction sector.

Human pose detection, especially joint angle calculation in a work environment, is crucial to early deduction of muscoloskeletal disorders. Conventional digital technology-based methods to detect pose flaws focus on location information from wearables and laboratory-controlled motion sensors. For the first time, this paper presents novel computer vision (recorded and live videos using mobile and embedded cameras) and digital image-related deep learning methods without attachment to the human body for manual handling pose deduction and analysis of angles, neckline and torso line in an actual construction work environment.

Vision, audition, olfactory, tactile and taste are five important senses that human uses to interact with the real world. As facing more and more complex environments, a sensing system is essential for intelligent robots with various types of sensors. To mimic human-like abilities, sensors similar to human perception capabilities are indispensable. However, most research only concentrated on analyzing literature on single-modal sensors and their robotics application.

This study presents a systematic review of five bioinspired senses, especially considering a brief introduction of multimodal sensing applications and predicting current trends and future directions of this field, which may have continuous enlightenments.

This review shows that bioinspired sensors can enable robots to better understand the environment, and multiple sensor combinations can support the robot’s ability to behave intelligently.

The review starts with a brief survey of the biological sensing mechanisms of the five senses, which are followed by their bioinspired electronic counterparts. Their applications in the robots are then reviewed as another emphasis, covering the main application scopes of localization and navigation, objection identification, dexterous manipulation, compliant interaction and so on. Finally, the trends, difficulties and challenges of this research were discussed to help guide future research on intelligent robot sensors.

The purpose of this research is to develop a generic framework of a digital twin (DT)-based automated construction progress monitoring through reality capture to extended reality (RC-to-XR).

IDEF0 data modeling method has been designed to establish an integration of reality capturing technologies by using BIM, DTs and XR for automated construction progress monitoring. Structural equation modeling (SEM) method has been used to test the proposed hypotheses and develop the skill model to examine the reliability, validity and contribution of the framework to understand the DRX model's effectiveness if implemented in real practice.

The research findings validate the positive impact and importance of utilizing technology integration in a logical framework such as DRX, which provides trustable, real-time, transparent and digital construction progress monitoring.

DRX system captures accurate, real-time and comprehensive data at construction stage, analyses data and information precisely and quickly, visualizes information and reports in a real scale environment, facilitates information flows and communication, learns from itself, historical data and accessible online data to predict future actions, provides semantic and digitalize construction information with analytical capabilities and optimizes decision-making process.

The research presents a framework of an automated construction progress monitoring system that integrates BIM, various reality capturing technologies, DT and XR technologies (VR, AR and MR), arraying the steps on how these technologies work collaboratively to create, capture, generate, analyze, manage and visualize construction progress data, information and reports.

Buildings and their use is a complex process from design to occupation. Buildings produce huge volumes of data such as building information modelling (BIM), sensor (e.g. from building management systems), occupant and building maintenance data. These data can be spread across multiple disconnected systems in numerous formats, making their combined analysis difficult. The purpose of this paper is to bring these sources of data together, to provide a more complete account of a building and, consequently, a more comprehensive basis for understanding and managing its performance.

Building data from a sample of newly constructed housing units were analysed, several properties were identified for the study and sensors deployed. A sensor agnostic platform for visualising real-time building performance data was developed.

Data sources from both sensor data and qualitative questionnaire were analysed and a matrix of elements affecting building performance in areas such as energy use, comfort use, integration with technology was presented. In addition, a prototype sensor visualisation platform was designed to connect in-use performance data to BIM.

This work presents initial findings from a post occupancy evaluation utilising sensor data. The work attempts to address the issues of BIM in-use scenarios for housing sector. A prototype was developed which can be fully developed and replicated to wider housing projects. The findings can better address how indoor thermal comfort parameters can be used to improve housing stock and even address elements such as machine learning for better buildings.

Using real-time support systems may help operators in road construction to improve paving and compaction operations. Nowadays, these systems transform from descriptive to prescriptive systems. Prescriptive or operator guidance systems propose operators actionable compaction strategies and guidance, based on the data collected. It is investigated how these systems mediate the perceptions and actions of operators in road pavement practice.

A case study is conducted on the specific application of an operator guidance system in a road pavement project. In this case study, comprehensive information is presented regarding the process of converting input in the form of data from cameras and sensors into useful output. The ways in which the operator guidance systems translate data into actionable guidance for operators are analyzed from the technological mediation perspective.

Operator guidance systems mediate actions of operators physically, cognitively and contextually. These different types of action mediation are related to preconditions for successful implementation and use of these systems. Coercive interventions only succeed if there is widespread agreement among the operators. Persuasive interventions are most effective when collective and individual interests align. Contextual influence relates to designs of the operator guidance systems that determine human-technology interactions when using them.

This is the first study that analyzes the functioning of an operator guidance system using the technological mediation approach. It adds a new perspective on the interaction between this system and its users in road pavement practice.

This study aims to consider active vision in low-visibility environments to reveal the factors of optical properties which affect visibility and to explore a method of obtaining different depths of fields by multimode imaging.Bad weather affects the driver’s visual range tremendously and thus has a serious impact on transport safety.

A new mechanism and a core algorithm for obtaining an excellent large field-depth image which can be used to aid safe driving is designed and implemented. In this mechanism, atmospheric extinction principle and field expansion system are researched as the basis, followed by image registration and fusion algorithm for the Infrared Extended Depth of Field (IR-EDOF) sensor.

The experimental results show that the idea we propose can work well to expand the field depth in a low-visibility road environment as a new aided safety-driving sensor.

The paper presents a new kind of active optical extension, as well as enhanced driving aids, which is an effective solution to the problem of weakening of visual ability. It is a practical engineering sensor scheme for safety driving in low-visibility road environments.