Tensorpose: Real-time pose estimation for interactive applications

摘要

The state of the art has outstanding results for 2D multi-person pose estimation using multi-stage Deep Neural Networks in images with high accuracy. However, the use of these models on real-time applications may be impractical not just because they are computationally intensive, but also because they suffer from flicking, from the inability for capturing temporal correlations among video frames, as well as from image degradation. To tackle these problems, we expand the use of pose estimation to motion capture in interactive applications. To do so, we propose a novel deep neural network with streamlined architecture and tensor decomposition for pose estimation with improved processing time, named TensorPose. We introduce an architecture for markerless motion capture using Convolutional Neural Networks combined with sparse optical flow and Kalman Filters. We also apply this architecture in a multi-user environment, based on the Holojam framework, where it is possible to create simultaneous collaborative experiences. (C) 2019 Published by Elsevier Ltd.