Recognition of workers’ actions from time-series signal images using deep convolutional neural network

Abstract

Purpose

Construction action recognition is essential to efficiently manage productivity, health and safety risks. These can be achieved by tracking and monitoring construction work. This study aims to examine the performance of a variant of deep convolutional neural networks (CNNs) for recognizing actions of construction workers from images of signals of time-series data.

Design/methodology/approach

This paper adopts Inception v1 to classify actions involved in carpentry and painting activities from images of motion data. Augmented time-series data from wearable sensors attached to worker's lower arms are converted to signal images to train an Inception v1 network. Performance of Inception v1 is compared with the highest performing supervised learning classifier, k-nearest neighbor (KNN).

Findings

Results show that the performance of Inception v1 network improved when trained with signal images of the augmented data but at a high computational cost. Inception v1 network and KNN achieved an accuracy of 95.2% and 99.8%, respectively when trained with 50-fold augmented carpentry dataset. The accuracy of Inception v1 and KNN with 10-fold painting augmented dataset is 95.3% and 97.1%, respectively.

Research limitations/implications

Only acceleration data of the lower arm of the two trades were used for action recognition. Each signal image comprises 20 datasets.

Originality/value

Little has been reported on recognizing construction workers' actions from signal images. This study adds value to the existing literature, in particular by providing insights into the extent to which a deep CNN can classify subtasks from patterns in signal images compared to a traditional best performing shallow network.

Citation

Ogunseiju, O.R., Olayiwola, J., Akanmu, A.A. and Nnaji, C. (2022), "Recognition of workers’ actions from time-series signal images using deep convolutional neural network", Smart and Sustainable Built Environment, Vol. 11 No. 4, pp. 812-831. https://doi.org/10.1108/SASBE-11-2020-0170