Deep reinforcement learning-based attitude motion control for humanoid robots with stability constraints

Abstract

Purpose

This paper aims to present an intelligent motion attitude control algorithm, which is used to solve the poor precision problems of motion-manipulation control and the problems of motion balance of humanoid robots. Aiming at the problems of a few physical training samples and low efficiency, this paper proposes an offline pre-training of the attitude controller using the identification model as a priori knowledge of online training in the real physical environment.

Design/methodology/approach

The deep reinforcement learning (DRL) of continuous motion and continuous state space is applied to motion attitude control of humanoid robots and the robot motion intelligent attitude controller is constructed. Combined with the stability analysis of the training process and control process, the stability constraints of the training process and control process are established and the correctness of the constraints is demonstrated in the experiment.

Findings

Comparing with the proportion integration differentiation (PID) controller, PID + MPC controller and MPC + DOB controller in the humanoid robots environment transition walking experiment, the standard deviation of the tracking error of robots’ upper body pitch attitude trajectory under the control of the intelligent attitude controller is reduced by 60.37 per cent, 44.17 per cent and 26.58 per cent.

Originality/value

Using an intelligent motion attitude control algorithm to deal with the strong coupling nonlinear problem in biped robots walking can simplify the control process. The offline pre-training of the attitude controller using the identification model as a priori knowledge of online training in the real physical environment makes up the problems of a few physical training samples and low efficiency. The result of using the theory described in this paper shows the performance of the motion-manipulation control precision and motion balance of humanoid robots and provides some inspiration for the application of using DRL in biped robots walking attitude control.

Citation

Shi, Q., Ying, W., Lv, L. and Xie, J. (2020), "Deep reinforcement learning-based attitude motion control for humanoid robots with stability constraints", Industrial Robot, Vol. 47 No. 3, pp. 335-347. https://doi.org/10.1108/IR-11-2019-0240