Spotting celebrities among peers in a TV show: how to exploit web querying for weakly supervised visual diarization

摘要

In this paper, we propose a novel solution for popularity recognition. This methodology consists of categorizing and exploiting web image resources of people in terms of relevant identities. To demonstrate its usefulness, we also study the effects of incorporating this procedure into a visual diarization system.In our setting, training data is obtained by querying Google Images about the known identities of the participants in a TV show (we only know who participates but not when). As Google queries may return imprecise results, images retrieved for each query (or identity) are processed to distinguish true from false positives, keeping the former while filtering out the latter.Next, facial clustering is performed to drive a filtering process discarding noisy samples (false positives) from returned images (i.e. only images linked to the principal cluster are adopted as training data).For popularity recognition, Random Forest (RF) and support vector machine (SVM) classifiers have been tested. Three different types of features have been proposed to build the models upon: image related features, query related features and clustering related features.Feature ranking and feature selection techniques applied show that clustering related features are the most important for popularity recognition. In fact, the RF model based on the 3 top ranked features achieves an accuracy close to 100% on test set. Results also demonstrate that integrating the popularity solution into our visual diarization pipeline helps to reduce the Diarization Error Rate (DER) in a 2%, removing around a 15% of noisy identities, which confirms the quality of this procedure and its high performance in other scenarios.