Framework for evaluation of sound event detection in web videos

摘要

The largest source of sound events is web videos. Most videos lack soundevent labels at segment level, however, a significant number of them do respondto text queries, from a match found to their metadata by the search engine. Inthis paper we explore the extent to which a search query could be used as thetrue label for the presence of sound events in the videos. For this, wedeveloped a framework for large-scale sound event recognition on web videos.The framework crawls videos using search queries corresponding to 78 soundevent labels drawn from three datasets. The datasets are used to train threeclassifiers, which were then run on 3.7 million video segments. We evaluatedperformance using the search query as the true label and compare it (on asubset) with human labeling. Both types exhibited close performance, to within10%, and similar performance trends as the number of evaluated segmentsincreased. Hence, our experiments show potential for using search query as apreliminary true label for sound events in web videos.