Baseball Player Behavior Recognition System using Multimodal Features with an Augmented Reality Display on a Smart Glass

摘要

In this paper, a real-time baseball player behavior recognition system is proposed. By analyzing the sensing signals from the wearable sensors and the skeletons from the depth channel belonging a Kinect camera, the behaviors can be recognized by the proposed system. When a body part is occluded or the depth frames is with motion blur effects in a depth camera, the sensing signals from worn sensors can compensate the recognition capability. In addition, by analyzing the multimodal features obtained from heterogeneous sensors, the recognized results can be displayed on a smart glass with an augmented reality displaying. In this prototype, a player's behavior can be monitored by a coach to assist the advising process in an on-field and off-field baseball playing environment.