Deeply-learned and spatial-temporal feature engineering for human action understanding

摘要

Accurately recognizing various human actions is a key technique in many AI applications, such as visual tracking and human-computer interaction. Aiming at solving the difficulty that the local spatial-temporal features at low-layer is limited and the descriptiveness of middle-level features is weak, we propose a novel human action understanding framework by leveraging the spatial-temporal depth features. More specifically, based on the fact that violent action regions encode highly discriminative information during human action recognition, we employ human depth clue of video images to identify the salient regions for each human. The optical flow characteristics in the regions are utilized as the energy function to measure the regional representativeness. The deep learning architecture is proposed to identify the regions of motions by the energy function. In this way, the sample points are distributed in the areas with intense movements. The collected sample points are utilized as the learned features to capture human action. Based on the learned deep feature, an SVM classifier is used to identify different human action. Comprehensive experimental results have shown that the average recognition accuracy of our human action recognition algorithm reaches 92%, and also exhibits a high robustness to complicated backgrounds.