Ensemble One-dimensional Convolution Neural Networks for Skeleton-based Action Recognition

摘要

In this paper, we proposed a effective but extensible residualone-dimensional convolution neural network as base network, based on the thisnetwork, we proposed four subnets to explore the features of skeleton sequencesfrom each aspect. Given a skeleton sequences, the spatial information areencoded into the skeleton joints coordinate in a frame and the temporalinformation are present by multiple frames. Limited by the skeleton sequencerepresentations, two-dimensional convolution neural network cannot be useddirectly, we chose one-dimensional convolution layer as the basic layer. Eachsub network could extract discriminative features from different aspects. Ourfirst subnet is a two-stream network which could explore both temporal andspatial information. The second is a body-parted network, which could gainmicro spatial features and macro temporal features. The third one is anattention network, the main contribution of which is to focus the key framesand feature channels which high related with the action classes in a skeletonsequence. One frame-difference network, as the last subnet, mainly processesthe joints changes between the consecutive frames. Four subnets ensembletogether by late fusion, the key problem of ensemble method is each subnetshould have a certain performance and between the subnets, there are diversityexisting. Each subnet shares a wellperformance basenet and differences betweensubnets guaranteed the diversity. Experimental results show that the ensemblenetwork gets a state-of-the-art performance on three widely used datasets.