To read this content please select one of the options below:

Predictive visual control network for occlusion solution in human-following robot

Juncheng Zou (University of Science and Technology Beijing, Beijing, China)

Assembly Automation

ISSN: 0144-5154

Article publication date: 26 February 2021

Issue publication date: 27 July 2021




The purpose of this paper is to propose a new video prediction-based methodology to solve the manufactural occlusion problem, which causes the loss of input images and uncertain controller parameters for the robot visual servo control.


This paper has put forward a method that can simultaneously generate images and controller parameter increments. Then, this paper also introduced target segmentation and designed a new comprehensive loss. Finally, this paper combines offline training to generate images and online training to generate controller parameter increments.


The data set experiments to prove that this method is better than the other four methods, and it can better restore the occluded situation of the human body in six manufactural scenarios. The simulation experiment proves that it can simultaneously generate image and controller parameter variations to improve the position accuracy of tracking under occlusions in manufacture.


The proposed method can effectively solve the occlusion problem in visual servo control.



This work was supported partially by the National Natural Science Foundation of China under Grant 91648205, 61627808 and 61702516, the National Key Research and Development Program of China (2017YFB1300200, 2017YFB1300203), Guangdong Key Area Research and Development Project 2018B010108003, Dongguan City Core Technology Research Frontier Project 2019622101001, Innovation Center of Robotics And Intelligent Equipment, KEY Laboratory of Robotics and Intelligent Equipment of Guangdong Regular Institutions of Higher Education (Grant No.2017KSYS009), the Beijing Natural Science Foundation No. L172052, the Strategic Priority Research Program of Chinese Academy of Science under Grant XDB32050100.


Zou, J. (2021), "Predictive visual control network for occlusion solution in human-following robot", Assembly Automation, Vol. 41 No. 2, pp. 187-199.



Emerald Publishing Limited

Copyright © 2021, Emerald Publishing Limited

Related articles