Leveraging Depth Cameras and Wearable Pressure Sensors for Full-body Kinematics and Dynamics Capture

摘要

We present a new method for full-body motion capture that usesrninput data captured by three depth cameras and a pair of pressuresensingrnshoes. Our system is appealing because it is low-cost,rnnon-intrusive and fully automatic, and can accurately reconstructrnboth full-body kinematics and dynamics data. We first introduce arnnovel tracking process that automatically reconstructs 3D skeletalrnposes using input data captured by three Kinect cameras and wearablernpressure sensors. We formulate the problem in an optimizationrnframework and incrementally update 3D skeletal poses withrnobserved depth data and pressure data via iterative linear solvers.rnThe system is highly accurate because we integrate depth data fromrnmultiple depth cameras, foot pressure data, detailed full-body geometry,rnand environmental contact constraints into a unified framework.rnIn addition, we develop an efficient physics-based motionrnreconstruction algorithm for solving internal joint torques and contactrnforces in the quadratic programming framework. During reconstruction,rnwe leverage Newtonian physics, friction cone constraints,rncontact pressure information, and 3D kinematic poses obtainedrnfrom the kinematic tracking process to reconstruct full-bodyrndynamics data. We demonstrate the power of our approach by capturingrna wide range of human movements and achieve state-of-theartrnaccuracy in our comparison against alternative systems.