A Data Reduction Procedure for 'Principal Cast' and other 'Talking Head' Detection

摘要

We describe a technique for reducing the data set for "principal cast" and other "talking head" detection in broadcast news content using the spatial attributes of MPEG-7 Motion Activity descriptor. The fact that these descriptors are easy to extract from compressed domain and also work well when used for matching talking head sequences, motivated us to utilize them for rapidly pruning the data set for subsequent sophisticated face detection techniques. We are thus able to speed up the process of finding the "principal cast" from broadcast news content by reducing the number of segments on which computationally more expensive face detection and recognition is employed. We present the experimental results of three clustering procedures using these descriptors on news content. The first procedure is based on a single template obtained from the centroid of ground truth set and is computationally less expensive. The second clustering procedure is based on multiple templates, which are the mean feature vectors of the component Gaussians of a Gaussian Mixture Model (GMM) trained best to fit the training data. The third clustering procedure is based on a HMM which captures the temporal interaction between the multiple templates. We are able to save 50% on computation measured in terms of number of rejected shots to total number of shots while missing 25% of talking head shots in the news program. We also observe that the second and third clustering procedure while being slightly computationally intensive allows for higher pruning factors with more accuracy.