Search results

Mixed reality is expanding in the industrial market and several companies in various fields are adapting this set of technologies for various purposes, such as optimizing processes, improving the programming tasks and promoting the interactivity of their products with the users, or even improving teaching or training. Robotics is another area that can benefit from these recent technologies. In fact, most of the current and futuristic robotic applications, namely, the areas related to advanced manufacturing tasks (e.g. additive-manufacturing, collaborative robotics, etc.), require new technics to actually perceive the result of several actions, including programming tasks, anticipate trajectories, visualize the motion and related information, interface with programmers and users and several other human–machine interfaces. Consequently, this paper aims to explain a new concept of human–machine interfaces aiming to improve the interaction between advanced users and industrial robotic work cells.

The presented concept uses two different applications (apps) developed to explore the advanced features of the Microsoft HoloLens device. The objectives of the project reported in this paper are to optimize robot paths, just by allowing the advanced user to adjust the selected path through the mixed reality environment, and create new paths, just by allowing the advanced user to insert points in the mixed reality environment, correct them as needed, connect them using a certain type of motion, parametrize them (in terms of velocity, motion precision, etc.) and command them to the robot controller.

The solutions demonstrated in this paper show how mixed reality can be used to allow users, with limited programming experience, to fully use the robotics fields. They also show clearly that the integration of the mixed reality technology in the current robot systems will be a turning point in reducing the complexity for end-users.

There are two challenges in the developed applications. The first relates to the robot tool identification, which is very sensitive to lighting conditions or to very complex robot tools. This can result in positioning errors when the software shows the path in the mixed reality scene. The paper presents solutions to overcome this problem. Another unattended challenge is associated with handling the robot singularities when adjusting or creating new paths. Ongoing work is concentrated in creating mechanisms that prevent the end-user to create paths that contain unreachable points or paths that are not feasible because of bad motion parameters.

This paper demonstrates the utilization of mixed reality device to improve the tasks of programming and commanding manufacturing work cells based on industrial robots [see video in (Pires et al., 2018)]. As the presented devices and robot cells are the basis for Industry 4.0 objectives, this demonstration has a vast field of application in the near future, positively influencing the way complex applications, that require much close cooperation between humans and machines, are thought, planned and built. The paper presents two different applications fully ready to use in industrial environments. These applications are scientific experiments designed to demonstrate the principles and technologies of mixed reality applied to industrial robotics, namely, for improving the programming task.

Although the HoloLens device opens outstanding new areas for robot command and programming, it is still expensive and somehow heavy for everyday use. Consequently, this opens an opportunity window to combine these devices with other mobile devices, such as tablets and phones, building applications that take advantage of their combined features.

The paper presents two different applications fully ready to use in industrial environments. These applications are scientific experiments designed to demonstrate the principles and technologies of mixed reality applied to industrial robotics, namely, for improving the programming task. The first application is about path visualization, i.e. enables the user to visualize, in a mixed reality environment, any path preplanned for the robot cell. With this feature, the advanced user can follow the robot path, identify problems, associate any difficulty in the final product with a particular issue in the robot paths, anticipate execution problems with impact on the final product quality, etc. This is particularly important for not only advanced applications, but also for cases where the robot path results from a CAD package (in an offline fashion). The second application consists of a graphical path manipulation procedure that allows the advanced user to create and optimize a robot path. Just by exploring this feature, the end-user can adjust any path obtained from any programming method, using the mixed reality approach to guide (visually) the path manipulation procedure. It can also create a completely new path using a process of graphical insertion of point positions and paths into the mixed reality scene. The ideas and implementations of the paper are original and there is no other example in the literature applied to industrial robot programming.

Many manufacturers are exploring adopting smart technologies in their operations, also referred to as the shift towards “Industry 4.0”. Employees' contribution to high-tech initiatives is key to successful Industry 4.0 technology adoption, but few studies have examined the determinants of employee acceptance. This study, therefore, aims to explore how managers affect employees' acceptance of Industry 4.0 technology, and, in turn, Industry 4.0 technology adoption.

Rooted in the unified theory of acceptance and use of technology model and social exchange theory, this inductive research follows an in-depth comparative case study approach. The two studied Dutch manufacturing firms engaged in the adoption of Industry 4.0 technologies in their primary processes, including cyber-physical systems and augmented reality. A mix of qualitative methods was used, consisting of field visits and 14 semi-structured interviews with managers and frontline employees engaged in Industry 4.0 technology adoption.

The cross-case comparison introduces the manager's need to adopt a transformational leadership style for employees to accept Industry 4.0 technology adoption as an organisational-level factor that extends existing Industry 4.0 technology user acceptance theorising. Secondly, manager's and employee's recognition and serving of their own and others' emotions through emotional intelligence are proposed as an additional individual-level factor impacting employees' acceptance and use of Industry 4.0 technologies.

Synthesising these insights with those from the domain of Organisational Behaviour, propositions were derived from theorising the social aspects of effective Industry 4.0 technology adoption.

The purpose of this study is to develop a building information modelling (BIM)-based mixed reality (MR) application to enhance and facilitate the process of managing bridge inspection and maintenance works remotely from office. It aims to address the ineffective decision-making process on maintenance tasks from the conventional method which relies on documents and 2D drawings on visual inspection. This study targets two key issues: creating a BIM-based model for bridge inspection and maintenance; and developing this model in a MR platform based on Microsoft Hololens.

Literature review is conducted to determine the limitation of MR technology in the construction industry and identify the gaps of integration of BIM and MR for bridge inspection works. A new framework for a greater adoption of integrated BIM and Hololens is proposed. It consists of a bridge information model for inspection and a newly-developed Hololens application named “HoloBridge”. This application contains the functional modules that allow users to check and update the progress of inspection and maintenance. The application has been implemented for an existing bridge in South Korea as the case study.

The results from pilot implementation show that the inspection information management can be enhanced because the inspection database can be systematically captured, stored and managed through BIM-based models. The inspection information in MR environment has been improved in interpretation, visualization and visual interpretation of 3D models because of intuitively interactive in real-time simulation.

The proposed framework through “HoloBridge” application explores the potential of integrating BIM and MR technology by using Hololens. It provides new possibilities for remote inspection of bridge conditions.

The purpose of this paper is to provide an insight into how augmented reality (AR) technologies are being applied to robotics.

Following an introduction and a brief historical background to AR, this first provides examples of AR applications in robot programming. It then gives examples of recent research into AR-based robot teleoperation. Research activities involving the virtual fixtures (VF) technique are then discussed and finally, brief conclusions are drawn.

Because AR concepts were first investigated in the 1990s, applications involving robotics have been widely studied. Programming with the aid of AR devices, such as the HoloLens headset, can be simplified and AR methods, including the VF technique, can improve the accuracy and speed of teleoperation, manipulation and positional control tasks. They can also provide visual or haptic feedback which leads to more intuitive operation and significantly reduces the cognitive load on the operator.

This provides an insight into the growing role of AR in robotics by providing examples of recent research in a range of applications.

The possibility of integrating building information in an augmented reality (AR) environment provides an effective solution to all phases of a building's lifecycle. This paper explores the integration of building information modelling (BIM) and AR to effectively visualise building information models in an AR environment and evaluates the currently available AR tools.

A BIM model of a selected office room was created and superimposed to the actual physical space using two different AR devices and four different AR applications. The superimposing techniques, accuracy and the level of information that can be visualised were then investigated by performing a walk-through analysis.

From the investigation, it can be concluded that model positioning can be inaccurate depending on the superimposing method used and the AR device. Moreover, using the currently available techniques, only static building information can be superimposed and visualised in AR, showing a need to integrate data from Internet of Things (IoT) sensors into the current BIM-AR processes to allow visualisation of accurate and high-quality operational building information.

A practical process and method for visualising and superimposing BIM models in an AR environment have been described. Recommendations to improve superimposing accuracy are provided. The assessment of type, quality and level of detail that can be visualised indicates the areas that need improvement to increase the effectiveness of building information's visualisation in AR.

At present, teaching methods based on 2D drawings are still commonly used for educating students on the location of steel reinforcement bars in concrete. However, traditional teaching methods have limitations as students can find it difficult to understand 2D drawings. This study aims to develop an interactive and collaborative augmented reality environment (ICARE) using augmented reality (AR) technology to improve students' engagement in learning.

This study develops an ICARE prototype, which is organized into two stages: (1) The augmented teaching environment comprising of models and interactive components; (2) The AR collaborative application which uses Photon Unity Networking (PUN) plugin and Azure spatial anchors cloud service. The AR-based teaching environment runs with Universal Windows Platform (UWP) to enable development in the HoloLens 2 through Microsoft Visual Studio.

An experimental study was conducted, where 60 students were divided into three groups employing Drawings-based, building information modeling (BIM)-based and AR-based methods for teaching. After the test, the three groups of students were requested to complete a questionnaire. According to the analysis of the experimental results, the ICARE can improve students' comprehension, memory of learned materials and their ability to read and understand steel reinforcement drawings improving the quality of teaching, especially interactivity and engagement.

As illustrated in the experiments, the developed ICARE has outstanding performance over conventional approaches in civil engineering courses that can improve students' comprehension and memory of knowledge and their ability to read and understand steel bar drawings. This study provides empirical evidence that AR is a promising technology that can be integrated with traditional classroom instruction and can improve students' comprehension and memory of knowledge and their ability to read and understand steel bar drawings.

This chapter reports on research that explores new and emerging extended reality [XR] technologies and how they might provide opportunities to trial, investigate, and put into practice their potential to reverse processes of atomisation, polarisation, and intercultural discomfort, in our contemporary society. This transdisciplinary practice-led research was underpinned by disciplines of computer science and engineering, social sciences, history, diverse community economics, human ecology, and Indigenous psychology. The collaboration between these various disciplines with the Māori and non-Māori community members allowed researchers to understand current societal stressors, prioritise relationality, and explore our shared values in the creation of XR experiences for exhibition in the galleries, libraries, archives, and museums [GLAM] sector.

A discursive design framework motivated, inspired, provoked, persuaded, and reminded inspiring collaborators, and visitors to the exhibitions, the value of (re)connecting with people and overcoming interracial awkwardness through these curated experiences. The XR technologies provided women a platform to discuss and reimagine first encounters between people from different cultural backgrounds. The technologies included a 180° stereoscopic projection, Common Sense, in which Māori Elder Irene Hancy shared her insight about social engagement and haptic HONGI in which visitors were greeted by a Māori woman Tania Remana via augmented reality. This research has been motivated by a desire to promote and support intercultural understanding in Aotearoa New Zealand, and it extends research by other non-Māori and Māori scholars.

The deployment of assistive technologies affects well-being and productivity at industrial workplaces. Augmented reality (AR) is one of these technologies that has become increasingly deployed in manufacturing facilities to assist employees on the shopfloor. This paper aims to shed light on users’ experiences with AR-based assistance systems, specifically on the sense of autonomy users experience during an AR-assisted assembly task. Based on that, this paper draws implications for the design of future industrial workplaces to improve workers’ health, well-being and productivity.

The authors conducted a laboratory experiment with 117 participants. Within semi-structured interviews, the authors asked the participants about their general experience, as well as their sense of autonomy and responsibility.

The study results indicate a limited perception of autonomy. Connected to this, the participants took over a passive working attitude and experienced a limited sense of responsibility concerning the output of the AR-assisted assembly task. At the same time, however, the participants still attributed assembly errors internally.

AR-assistance holds both benefits and risks for worker’s health, well-being and productivity. With this study, the author aims to increase the understanding about the perception of autonomy and control at industrial workplaces. Thus, the authors conclude with design implications for developing and implementing assistive technologies in a way that beneficial effects for employees can be achieved.

This study aims to investigate “immersive reading,” which occurs when individuals read text while in a virtual reality (VR) or augmented reality (AR) environment.

In Experiment 1, 64 participants read text passages and answered multiple-choice questions in VR and AR head-mounted displays (HMDs) compared with doing the same task on liquid crystal display (LCD). In Experiment 2, 31 participants performed the same reading tasks but with two VR HMDs of different display quality.

Compared with reading on LCD as the baseline, participants reading in VR and AR HMDs got 82% (VR) and 88% (AR) of the information accurately. Participants tended to respond more accurately and faster, though not statistically significant, with the VR HMD of higher pixel density in the speed-reading task.

The authors observed the speed and accuracy of reading in VR and AR environments, compared with the reading speed and accuracy on an LCD monitor. The authors also compared the reading performance on two VR HMDs that differed in display quality but were otherwise similar in every way.

This column aims to take a brief look at wayfinding in the digital age, and explores several opportunities that are on the horizon for libraries that could enhance the user experience for their patrons.

This column provides a survey of the current views of wayfinding in the digital age.

There is new technology being developed that increasingly blurs the lines between the digital and the physical.

Devices such as the HoloLens and iBeacon, while nascent, will quickly become a standard for new approaches that people can use to interact with their environments integrating digital information. This affords libraries a unique opportunity over the course of the next 5-10 years to take advantage of this technology to assist patrons with their information needs.