Multiple Cue Integrated Action Detection

摘要

We present an action recognition scheme that integrates multiple modality of cues that include shape, motion and depth to recognize human gesture in the video sequences. In the proposed approach we extend classification framework that is commonly used in 2D object recognition to 3D spatio-temporal space for recognizing actions. Specifically, a boosting-based classifier is used that learns spatio-temporal features specific to target actions where features are obtained from temporal patterns of shape contour, optical flow and depth changes occurring at local body parts. The individual features exhibit different strength and sensitivity depending on many factors that include action, underlying body parts and background. In the current method, the multiple cues of different modalities are combined optimally by fisher linear discriminant to form a strong feature that preserve strength of individual cues. In the experiment, we apply the integrated action classifier on a set of target actions and evaluate its performance by comparing with single cue-based cases and present qualitative analysis of performance gain.