Tracking and frame-rate enhancement for real-time 2D human pose estimation

摘要

We propose a near real-time solution for frame-rate enhancement that enables the use of existing sophisticated pose estimation solutions at elevated frame rates. Our approach couples a keypoint human pose estimator with optical flow using a multistage system of queues operating in a multi-threaded environment. As additional contributions, we propose a pose tracking solution and an approach to overcome errors caused by optical flow. A reduction in error in the range of 30-35% is observed at practical frame rates of pose estimator (4-6 frames per second) while processing 1920x1080 resolution 30 frames-per-second videos at native frame rate. Slower frame rates have increased the reduction of error up to 50%, thereby promoting the use of cheaper hardware and sharing of expensive hardware. Thus, while improving accuracy by enabling sophisticated pose estimation models to operate at above-par frame rates, our approach reduces cost per frame by promoting efficient resource utilization.