Advanced
search

To read this content please select one of the options below:

Synchronous AR assembly assistance and monitoring system based on ego-centric vision

Xuyue Yin (Institute of Intelligent Manufacturing and Information Engineering, School of Mechanical Engineering, Shanghai Jiao Tong University, Shanghai, China)
Xiumin Fan (Institute of Intelligent Manufacturing and Information Engineering, School of Mechanical Engineering, Shanghai Jiao Tong University, Shanghai, China)
Wenmin Zhu (Institute of Intelligent Manufacturing and Information Engineering, School of Mechanical Engineering, Shanghai Jiao Tong University, Shanghai, China)
Rui Liu (Institute of Intelligent Manufacturing and Information Engineering, School of Mechanical Engineering, Shanghai Jiao Tong University, Shanghai, China)

Assembly Automation

ISSN: 0144-5154

Article publication date: 16 October 2018

Issue publication date: 16 April 2019

625

Abstract

Purpose

Aiming at presenting an interaction-free assembly assistance tool, the purpose of this paper is to propose a synchronous augmented reality (AR) assembly assistance and monitoring system. The system monitors operator’s hands activity and process completeness to recognize the assembly state, then display the AR contents contextually.

Design/methodology/approach

An assembly behavior recognition method is proposed based on gesture recognition. An assembly completeness inspection method is proposed based on SURF feature matching. Assembly state and AR display state are solved by a novel sequential hybrid AR display control strategy. A synchronous multi-channel AR view output strategy is proposed based on QR matrix decomposition.

Findings

A prototype system has been developed, and case study is performed on an industrial product. Experiments are performed to verify the feasibility, efficiency and recognition accuracy of the proposed methods.

Research limitations/implications

The proposed system assists users to perform assembly tasks with automatic visual guidance and vision monitoring, avoiding distractions caused by redundant human–computer interactions.

Practical implications

All methods are integrated to work on only one head-worn device, making the proposed system portable and cheaper. The vision processing pipelines and the view output channels are reconfigurable for customization.

Originality/value

This paper proposes an interaction-free AR assembly assistance and monitoring system. Assembly behavior recognition and assembly completeness inspection methods are integrated to monitor the assembly state. A sequential hybrid AR display control strategy is proposed to contextually update the AR contents. A synchronous multi-channel AR view output strategy is proposed to fulfill different visualization needs.

Keywords

Acknowledgements

This research is sponsored by Shanghai Aerospace Science and Technology Corporation (Grant NO. USCAST2015-17), which is greatly acknowledged. The authors are grateful to the editors and anonymous reviewers for their valuable comments.

Citation

Yin, X., Fan, X., Zhu, W. and Liu, R. (2019), "Synchronous AR assembly assistance and monitoring system based on ego-centric vision", Assembly Automation, Vol. 39 No. 1, pp. 1-16. https://doi.org/10.1108/AA-03-2017-032

Publisher

:

Emerald Publishing Limited

Copyright © 2018, Emerald Publishing Limited

Related articles

Manage cookies

We’re listening — tell us what you think

Join us on our journey