3-D body joint-specific HMM-based approach for human activity recognition from stereo posture image sequence

摘要

In this paper, a stereo camera-based novel approach for Human Activity Recognition (HAR) is presented using robust 3-D human body joint features and joint-specific Hidden Markov Models (HMMs). At first, body joint angles are estimated by co-registering a 3-D body model to the stereo video information (i.e., 3-D depth) of a human posture acquired by a stereo camera. Conventionally, all joint angles are augmented followed by discriminant feature extraction from them and a HMM is modeled for each activity. Although the traditional approach is straight forward and easy to implement but dependent to unnecessary joint features which are not even used in the activity. In this study, we focus on individual 3-D body joints rather than all joints together and body joint motion information in next frame is also considered in addition to the degree of freedom values (i.e., joint angles in current frame) of a joint. We propose a new way of modeling human activities and derive joint-specific HMMs. Based on motion information of the joints in next frame and degree of freedom information of body joints in the time-sequential distinguished activity video frames, the different activity classes are determined first. Each joint features are then mapped into codewords to generate a sequence of discrete symbols for joint-specific HMM. Then, joint-specific HMMs are trained according to their use in different activities. For testing, after determining the activity class based on the time-sequential body joint features, the discrete symbol sequence from each joint is applied to the trained joint-specific HMMs of the activities from that class only. Thus, for all body joints, the likelihoods of all activities are obtained by applying all body joint features and then, likelihoods for corresponding activities are summed up. Finally, one activity has been chosen with the highest likelihood from the summed likelihoods. Using joint-specific HMMs (i.e., multiple HMMs for an activity based on active body joints), superior recognition performance is obtained than the augmented joint angle feature-based single HMM for an activity as well as the traditional silhouette-based approaches.