Modeling human-skeleton motion patterns using conditional deep Boltzmann machine

摘要

This paper addresses the problem of modeling long-range motion patterns of a 3D human skeleton performing an activity. This problem is important, as such a model can be used in many applications, including person tracking via 3D pose estimation, and probabilistic sampling of realistic 3D skeleton sequences conducting different activities with different motion styles. To this end, we formulate a new generative model, called conditional deep Boltzmann machine (CDBM). CDBM defines a joint distribution of two hidden layers and 3D-skeleton pose predictions in the near future given human skeleton observations from the recent past. Our CDBM extends the conditional restricted Boltzmann machine (CRBM) and the factored conditional restricted Boltzmann machine (FCRBM) by introducing an additional hidden layer and removing the style layer, while preserving the computational efficiency of CRBM and FCRBM. The new hidden variables are aimed at capturing long-range and high-order spatiotemporal interactions among human body joints, and thus enable CDBM to effectively model 3D motion sequences with different activities and motion styles with a single set of parameters. Our experiments on the benchmark Motion Capture and HumanEva datasets demonstrate that our CDBM outperforms CRBM and achieves on par performance with FCRBM both in 3D pose based person tracking and realistic 3D skeleton sequence generating.