Human behavior recognition under occlusion based on two-stream network combined with BiLSTM

摘要

Human behavior recognition is an important subject in the field of computer vision, and it is a difficulty to recognize human behavior when the core part of human movement is blocked at some time of video. In order to solve the problem of occlusion in the process of human behavior recognition, this paper proposes a two-stream network structure incorporating bi-directional long and short time memory network (BiLSTM). A BiLSTM network with two branches is used to analyze the spatial feature information of each frame better. In order to make full use of the information of long distance optical flow and extract the variation information of sample characteristics on optical flow, a discriminant loss function of optical flow is proposed. The validity of the proposed method is verified by experimental comparison with other methods on the data set of occlusion behavior. The recognition accuracy of this method is up to 78.6%, and it has good robustness to the occlusion environment.